Estimating causal effects: considering three alternatives to difference-in-differences estimation

How do family doctors respond to reduced waiting times for cancer diagnosis in secondary care?

Reducing waiting times is a priority in public health systems. Efforts of healthcare providers to shorten waiting times could be negated if they simultaneously induce substantial increases in demand. However, separating out the effects of changes in supply and demand on waiting times requires an exogenous change in one element. We examine the impact of a pilot programme in some English hospitals to shorten waiting times for urgent diagnosis of suspected cancer on family doctors' referrals. We examine referrals from 6,666 family doctor partnerships to 145 hospitals between 1st April 2012 and 31st March 2019. Five hospitals piloted shorter waiting times initiatives in 2017. Using continuous difference-in-differences regression, we exploit the pilot as a 'supply shifter' to estimate the effect of waiting times on referral volumes for two suspected cancer types: bowel and lung. The proportion of referred patients breaching two-week waiting times targets for suspected bowel cancer fell by 3.9 percentage points in pilot hospitals in response to the policy, from a baseline of 4.8%. Family doctors exposed to the pilot increased their referrals (demand) by 10.8%. However, the pilot was not successful for lung cancer, with some evidence that waiting times increased, and a corresponding reduction in referrals of -10.5%. Family doctor referrals for suspected cancer are responsive at the margin to waiting times. Healthcare providers may struggle to achieve long-term reductions in waiting times if supply-side improvements are offset by increases in demand.

Evaluation of Changes in Prices and Purchases Following Implementation of Sugar-Sweetened Beverage Taxes Across the US

Importance

Sugar-sweetened beverage (SSB) taxes are promoted as key policies to reduce cardiometabolic diseases and other conditions, but comprehensive analyses of SSB taxes in the US have been difficult because of the absence of sufficiently large data samples and methods limitations.

Objective

To estimate changes in SSB prices and purchases following SSB taxes in 5 large US cities.

Design, Setting, and Participants

In this cross-sectional study with an augmented synthetic control analysis, changes in prices and purchases of SSBs were estimated following SSB tax implementation in Boulder, Colorado; Philadelphia, Pennsylvania; Oakland, California; Seattle, Washington; and San Francisco, California. Changes in SSB prices (in US dollars) and purchases (volume in ounces) in these cities in the 2 years following tax implementation were estimated and compared with control groups constructed from other cities. Changes in adjacent, untaxed areas were assessed to detect any increase in cross-border purchases. Data used for this analysis spanned from January 1, 2012, to February 29, 2020, and were analyzed between June 1, 2022, and September 29, 2023.

Main Outcomes and Measures

The main outcomes were the changes in SSB prices and volume purchased.

Results

Using nutritional information, 5500 unique universal product codes were classified as SSBs, according to tax designations. The sample included 26 338 stores-496 located in treated localities, 1340 in bordering localities, and 24 502 in the donor pool. Prices of SSBs increased by an average of 33.1% (95% CI, 14.0% to 52.2%; P < .001) during the 2 years following tax implementation, corresponding to an average price increase of 1.3¢ per oz and a 92% tax pass-through rate from distributors to consumers. SSB purchases declined in total volume by an average of 33.0% (95% CI, -2.2% to -63.8%; P = .04) following tax implementation, corresponding to a -1.00 price elasticity of demand. The observed price increase and corresponding volume decrease immediately followed tax implementation, and both outcomes were sustained in the months thereafter. No evidence of increased cross-border purchases following tax implementation was found.

Conclusions and Relevance

In this cross-sectional study, SSB taxes led to substantial, consistent declines in SSB purchases across 5 taxed cities following price increases associated with those taxes. Scaling SSB taxes nationally could yield substantial public health benefits.

General Practitioners activity patterns: the medium-term impacts of Primary Care Teams in France

Faced with the fragmentation of the French primary care system, public policies aim to promote multiprofessional teamwork to improve both delivery efficiency and health professionals' working conditions. Thus, a practice-level add-on payment backed by cooperation commitments is implemented to foster and sustain the development of multiprofessional primary care groups (MPCGs). We study the impact of practising in MPCGs for general practitioners (GPs) in terms of the supply of care, practice patterns and income. Based on this quasiexperimental framework with a panel dataset covering the period 2005-2017, we account for the selection into MPCGs by combining a difference-in-differences design with propensity score matching to prebalance samples. We show that GPs in MPCGs increased their patient list more rapidly than control GPs (+10% increase of encountered patients) without increasing their provision of services (number of visits and drug prescriptions) more rapidly. Instead, compared to control GPs, MPCG GPs had a significantly faster reduction in the average number of visits (+5.5% reduction) and the euro-amounts of drug prescriptions per patient (+7.2% reduction) and other prescriptions. The growth of these effects between the short and medium term moreover suggests that the properties of multi-professional coordination and cooperation need time to develop.

Vertical integration of general practices with acute hospitals in England: rapid impact evaluation

Background

Vertical integration means merging organisations that operate at different stages along the patient pathway. We focus on acute hospitals running primary care medical practices. Evidence is scarce concerning the impact on use of health-care services and patient experience.

Objectives

To assess the impact of vertical integration on use of hospital services, service delivery and patient experience and whether patients with multiple long-term conditions are affected differently from others.

Design

Rapid, mixed methods evaluation with four work packages: (1) review of NHS trust annual reports and other sources to understand the scale of vertical integration across England; (2) development of the statistical analysis; (3) analysis of national survey data on patient experience, and national data on use of hospital services over the 2 years preceding and following vertical integration, comparing vertically integrated practices with a variety of control practices; and (4) focus groups and interviews with staff and patients across three case study sites to explore the impact of vertical integration on patient experience of care.

Results

At 31 March 2021, 26 NHS trusts were in vertically integrated organisations, running 85 general practices across 116 practice sites. The earliest vertical integration between trusts and general practices was in 2015; a mean of 3.3 practices run by each trust (range 1-12). On average, integrated practices have fewer patients, are slightly more likely to be in the most deprived decile of areas, are more likely to hold an alternative provider medical services contract and have worse Quality and Outcomes Framework scores compared with non-integrated practices. Vertical integration is associated with statistically significant, modest reductions in rates of accident and emergency department attendances: 2% reduction (incidence rate ratio 0.98, 95% confidence interval 0.96 to 0.99; p < 0.0001); outpatient attendances: 1% reduction (incidence rate ratio 0.99, 95% confidence interval 0.99 to 1.00; p = 0.0061), emergency inpatient admissions: 3% reduction (incidence rate ratio 0.97, 95% confidence interval 0.95 to 0.99; p = 0.0062) and emergency readmissions: 5% reduction (incidence rate ratio 0.95, 95% confidence interval 0.91 to 1.00; p = 0.039), with no impact on length of stay, overall inpatient admissions or inpatient admissions for ambulatory care sensitive conditions. The falls in accident and emergency department and outpatient attendance rates are temporary. Focus groups and interviews with staff (N = 22) and interviews with patients (N = 14) showed that with vertical integration, health service improvements are introduced following a period of cultural interchange. Patients with multiple long-term conditions continue to encounter 'navigation work' choosing and accessing health-care provision, with diminishing continuity of care.

Limitations

In the quantitative analysis, we could not replicate the counterfactual of what would have happened in those specific locations had practices not merged with trusts. There was imbalance across three case study sites with regard to staff and patients recruited for interview, and the latter were drawn from patient participation groups who may not be representative of local populations.

Conclusions

Vertical integration can lead to modest reductions in use of hospital services and has minor or no impact on patient experience of care. Our analysis does not reveal a case for widespread roll-out of the approach.

Future research

Further quantitative follow-up of the longer-term impact of vertical integration on hospital usage and more extensive interviewing of patients and their carers about patient experiences of navigating care.

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (BRACE Project no. 16/138/31) and will be published in full in Health and Social Care Delivery Research; Vol. 11, No. 17. See the NIHR Journals Library website for further project information.

Mass gatherings for political expression had no discernible association with the local course of the COVID-19 pandemic in the USA in 2020 and 2021

Epidemic disease can spread during mass gatherings. We assessed the impact of a type of mass gathering about which comprehensive data were available on the local-area trajectory of the COVID-19 epidemic. Here we examined five types of political event in 2020 and 2021: the US primary elections, the US Senate special election in Georgia, the gubernatorial elections in New Jersey and Virginia, Donald Trump's political rallies and the Black Lives Matter protests. Our study period encompassed over 700 such mass gatherings during multiple phases of the pandemic. We used data from the 48 contiguous states, representing 3,108 counties, and we implemented a novel extension of a recently developed non-parametric, generalized difference-in-difference estimator with a (high-quality) matching procedure for panel data to estimate the average effect of the gatherings on local mortality and other outcomes. There were no statistically significant increases in cases, deaths or a measure of epidemic transmissibility (Rt) in a 40-day period following large-scale political activities. We estimated small and statistically non-significant effects, corresponding to an average difference of -0.0567 deaths (95% CI = -0.319, 0.162) and 8.275 cases (95% CI = -1.383, 20.7) on each day for counties that held mass gatherings for political expression compared to matched control counties. In sum, there is no statistical evidence of a material increase in local COVID-19 deaths, cases or transmissibility after mass gatherings for political expression during the first 2 years of the pandemic in the USA. This may relate to the specific manner in which such activities are typically conducted.

Impact of soda tax on beverage price, sale, purchase, and consumption in the US: a systematic review and meta-analysis of natural experiments

Background

As a primary source of added sugars in the US diet, sugar-sweetened beverage (SSB) consumption is presumed to contribute to obesity prevalence and poor oral health. We systematically synthesized and quantified evidence from US-based natural experiments concerning the impact of SSB taxes on beverage prices, sales, purchases, and consumption.

Methods

A keyword and reference search was performed in PubMed, Web of Science, Cochrane Library, Scopus, and EconLit from the inception of an electronic bibliographic database to Oct 31, 2022. Meta-analysis was conducted to estimate the pooled effect of soda taxes on SSB consumption, prices, passthrough rate, and purchases.

Results

Twenty-six natural experiments, all adopting a difference-in-differences approach, were included. Studies assessed soda taxes in Berkeley, Oakland, and San Francisco in California, Philadelphia in Pennsylvania, Boulder in Colorado, Seattle in Washington, and Cook County in Illinois. Tax rates ranged from 1 to 2 ¢/oz. The imposition of the soda tax was associated with a 1.06 ¢/oz. (95% confidence interval [CI] = 0.90, 1.22) increase in SSB prices and a 27.3% (95% CI = 19.3, 35.4%) decrease in SSB purchases. The soda tax passthrough rate was 79.7% (95% CI = 65.8, 93.6%). A 1 ¢/oz. increase in soda tax rate was associated with increased prices of SSBs by 0.84 ¢/oz (95% CI = 0.33, 1.35).

Conclusion

Soda taxes could be effective policy leverage to nudge people toward purchasing and consuming fewer SSBs. Future research should examine evidence-based classifications of SSBs, targeted use of revenues generated by taxes to reduce health and income disparities, and the feasibility of redesigning the soda tax to improve efficiency.

The impact of devolution on experienced health and well-being

Devolution of health systems from national to local levels is a common focus of policymakers across the world. The overarching aim is to improve population health by better meeting the specific needs of local citizens. We examine the case of a coordinated devolution across several public service sectors in Greater Manchester, England, in 2016. We estimate the impact on experienced health and well-being using Short-Form 12 scores from 13,938 adult respondents to the UK Household Longitudinal Survey between 2012 and 2020. We use difference-in-differences and lagged-dependent variable regressions to compare Greater Manchester to the rest of England. We find no statistically significant changes in experienced health and well-being over the four years following the start of devolution. Our findings suggest that devolving population health management alone without budgetary powers and local accountability mechanisms may not be effective in improving experienced health and well-being in the relatively short-term.

Dealing with confounding in observational studies: A scoping review of methods evaluated in simulation studies with single-point exposure

The aim of this article was to perform a scoping review of methods available for dealing with confounding when analyzing the effect of health care treatments with single-point exposure in observational data. We aim to provide an overview of methods and their performance assessed by simulation studies indexed in PubMed. We searched PubMed for simulation studies published until January 2021. Our search was restricted to studies evaluating binary treatments and binary and/or continuous outcomes. Information was extracted on the methods' assumptions, performance, and technical properties. Of 28,548 identified references, 127 studies were eligible for inclusion. Of them, 84 assessed 14 different methods (ie, groups of estimators that share assumptions and implementation) for dealing with measured confounding, and 43 assessed 10 different methods for dealing with unmeasured confounding. Results suggest that there are large differences in performance between methods and that the performance of a specific method is highly dependent on the estimator. Furthermore, the methods' assumptions regarding the specific data features also substantially influence the methods' performance. Finally, the methods result in different estimands (ie, target of inference), which can even vary within methods. In conclusion, when choosing a method to adjust for measured or unmeasured confounding it is important to choose the most appropriate estimand, while considering the population of interest, data structure, and whether the plausibility of the methods' required assumptions hold.

Does the US regional greenhouse gas initiative affect green innovation?

This study measures the impact of the implementation of the Regional Greenhouse Gas Initiative (RGGI) on firms' green innovation initiatives. We used 20 years of panel data from the Fortune 500 list of the US largest companies. Based on DID, a benchmark regression, the RGGI has a significant adverse effect on the green innovation of Fortune 500 companies, and we verified these findings with multiple robustness tests. As we investigate how energy-intensive industries were affected by RGGI, we found that it slowed down green innovation, but it was not statistically significant. This study provides a novel perspective on how the RGGI influences green innovation in firms and how different types of sectors respond to the policy. The findings indicate that the "weak" Porter Hypothesis has not been confirmed in the present carbon trading market (particularly the RGGI) for Fortune 500 firms in the USA. In terms of policy, we believe that a well-covered and differentiated legislation that fosters green innovation while being realistic about the policy's goal and the firm's environmental attitude, like emissions reduction through green innovation, is essential.

Differences in county-level cardiovascular disease mortality rates due to damage caused by hurricane Matthew and the moderating effect of social capital: a natural experiment

BACKGROUND

As the climate continues to warm, hurricanes will continue to increase in both severity and frequency. Hurricane damage is associated with cardiovascular events, but social capital may moderate this relationship. Social capital is a multidimensional concept with a rich theoretical tradition. Simply put, social capital refers to the social relationships and structures that provide individuals with material, financial, and emotional resources throughout their lives. Previous research has found an association between high levels of social capital and lower rates of cardiovascular (CVD) mortality. In post-disaster settings, social capital may protect against CVD mortality by improving access to life-saving resources. We examined the association between county-level hurricane damage and CVD mortality rates after Hurricane Matthew, and the moderating effect of several aspects of social capital and hurricane damage on this relationship. We hypothesized that (1) higher (vs. lower) levels of hurricane damage would be associated with increased CVD mortality rates and (2) in highly damaged counties, higher (vs. lower) levels of social capital would be associated with lower CVD mortality.

METHODS

Analysis used yearly (2013-2018) county-level sociodemographic and epidemiological data (n = 183). Sociodemographic data were compiled from federal surveys before and after Hurricane Matthew to construct, per prior literature, a social capital index based on four dimensions of social capital (sub-indices): family unity, informal civil society, institutional confidence, and collective efficacy. Epidemiological data comprised monthly CVD mortality rates constructed from monthly county-level CVD death counts from the CDC WONDER database and the US Census population estimates. Changes in CVD mortality based on level of hurricane damage were assessed using regression adjustment. We used cluster robust Poisson population average models to determine the moderating effect of social capital on CVD mortality rates in both high and low-damage counties.

RESULTS

We found that mean levels of CVD mortality increased (before and after adjustment for sociodemographic controls) in both low-damage counties (unadjusted. Mean = 2.50, 95% CI [2.41, 2.59], adjusted mean = 2.50, 95% CI [2.40, 2.72]) and high-damage counties (mean = 2.44, CI [2.29, 2.46], adj. Mean = 2.51, 95% CI [2.49, 2.84]). Among the different social capital dimensions, institutional confidence was associated with reduced initial CVD mortality in low-damage counties (unadj. IRR 1.00, 95% CI [0.90, 1.11], adj. IRR 0.91 CI [0.87, 0.94]), but its association with CVD mortality trends was null. The overall effects of social capital and its sub-indices were largely nonsignificant.

CONCLUSION

Hurricane damage is associated with increased CVD mortality for 18 months after Hurricane Matthew. The role of social capital remains unclear. Future research should focus on improving measurement of social capital and quality of hurricane damage and CVD mortality data.

Labor supply and informal care responses to health shocks within couples: Evidence from the UK

Shocks to health have been shown to reduce labor supply for the individual affected. Less is known about household self-insurance through a partner's response. Previous studies have presented inconclusive empirical evidence on the existence of a health-related Added Worker Effect, and results limited to labor and income responses. We use UK longitudinal data to investigate within households both the labor supply and informal care responses of an individual to the event of an acute health shock to their partner. Relying on the unanticipated timing of shocks, we combine Coarsened Exact Matching and Entropy Balancing algorithms with parametric analysis and exploit lagged outcomes to remove bias from observed confounders and time-invariant unobservables. We find no evidence of a health-related Added Worker Effect but a significant and sizable Informal Carer Effect. This holds irrespective of spousal labor market position or household financial status and ability to purchase formal care provision, suggesting that partners' substitute informal care provision for time devoted to leisure activities.

A comparison of four quasi-experimental methods: an analysis of the introduction of activity-based funding in Ireland

Background

Health services research often relies on quasi-experimental study designs in the estimation of treatment effects of a policy change or an intervention. The aim of this study is to compare some of the commonly used non-experimental methods in estimating intervention effects, and to highlight their relative strengths and weaknesses. We estimate the effects of Activity-Based Funding, a hospital financing reform of Irish public hospitals, introduced in 2016.

Methods

We estimate and compare four analytical methods: Interrupted time series analysis, Difference-in-Differences, Propensity Score Matching Difference-in-Differences and the Synthetic Control method. Specifically, we focus on the comparison between the control-treatment methods and the non-control-treatment approach, interrupted time series analysis. Our empirical example evaluated the length of stay impact post hip replacement surgery, following the introduction of Activity-Based Funding in Ireland. We also contribute to the very limited research reporting the impacts of Activity-Based-Funding within the Irish context.

Results

Interrupted time-series analysis produced statistically significant results different in interpretation, while the Difference-in-Differences, Propensity Score Matching Difference-in-Differences and Synthetic Control methods incorporating control groups, suggested no statistically significant intervention effect, on patient length of stay.

Conclusion

Our analysis confirms that different analytical methods for estimating intervention effects provide different assessments of the intervention effects. It is crucial that researchers employ appropriate designs which incorporate a counterfactual framework. Such methods tend to be more robust and provide a stronger basis for evidence-based policy-making.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08657-0.

Inferring heterogeneous treatment effects of work zones on crashes

The increasing number of work zone crashes has been a significant concern for road users, transportation agencies, and researchers. Crashes can be caused by work zones, and this effect changes across different work zone configurations, traffic volumes, roadway functional classifications, and weather conditions. This is typically represented by Crash Modification Functions (CMFunctions). However, current methods for developing work zone CMFunctions have two major limitations: (1) They focus on analyzing statistical associations and fail to mitigate the confounding bias due to possible unobservable roadway characteristics; and (2) They cannot address CMFunctions of multiple variables simultaneously, such as weather and traffic conditions, since they are represented using mixed data types (continuous and categorical) that could potentially affect the causal effect of work zones on crashes. In this study, we develop a method that utilizes causal forest with fixed-effect modeling to mitigate the confounding bias while identifying CMFunctions conditioning on various environmental characteristics, including work zone configurations, traffic volume, roadway functional classification, and weather conditions. The developed method was applied to 3378 work zones that occurred in Pennsylvania between 2015 and 2017. The results were validated via a series of robustness tests. The validations demonstrate that this method can mitigate the confounding bias and identify CMFunctions of multiple variables. The results also show that the causal effect of a work zone on crash occurrence is significantly positive (p<0.05) on roadways with high traffic volumes (e.g., > 20,000 vehicles per day) and on medium length (e.g., 2000 to 5000 m) work zones. It appears that having medium-long (e.g., between 6000 and 8000 m) work zones or long duration (e.g., longer than 4 h) work zones do not necessarily lead to extra crashes.

Employment trends at older ages: policy impact or secular change?

Observed increases in retirement age are generally attributed to policies to extend working lives (PEW). In a quasi-experimental design, we examine to what extent increases in employment of older workers can be attributed to secular changes in individual characteristics as opposed to PEW. We compare two countries: one with clear PEW (the Netherlands) and one without PEW (Norway). Data come from the Dutch Longitudinal Aging Study Amsterdam and the NORwegian Longitudinal study on Aging and Generations. From each study, two same-age (55-64 years) samples are selected, one recruited in 2002-03, and one recruited after five (Norway) and ten years (Netherlands). In pooled regression analysis, paid work is the outcome variable, and time of measurement, the main independent variable. Individual characteristics include age, sex, educational level, self-perceived health, functional limitations, sense of mastery, and work status of partner. Employment rose in both countries, faster in the Netherlands than in Norway. Of the rise in employment, individual characteristics explained less in the Netherlands than in Norway. Accounting for these, the interaction country*time was significant, indicating an extra rise in employment of 5.2 and 7.5% points for Dutch men and women, net of individual characteristics and unobserved factors that are assumed to be similar in both countries. The extra rise in the Netherlands represents 57% of the total rise for both sexes. Thus, secular change in individual characteristics explains part of the rise in employment in both countries. In the Netherlands, other factors such as PEW may additionally explain the rise in employment.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10433-021-00664-0.

The Effect of Biosimilar Prescription Targets for Erythropoiesis-Stimulating Agents on the Prescribing Behavior of Physicians in Germany

OBJECTIVES

This study aimed to aid decision makers by analyzing the impact of introducing biosimilar prescription targets on physician prescribing behavior in the prescription of biologic erythropoiesis-stimulating agents in Germany.

METHODS

We combined secondary data of regional level biosimilar prescription targets and secondary data of routinely collected claims data of dispensed prescriptions by physicians operating within the statutory health insurance system in ambulatory care across 7 German regions from 2009 to 2015. Two-way fixed-effects regression analysis was used to identify the average treatment effect of introducing biosimilar prescription targets at the physician level. The main outcome of interest was the share of biosimilar prescriptions on all prescriptions within the substance group. We compared 6 regions that introduced biosimilar prescription targets with 1 region without any prescription target policy.

RESULTS

Introducing biosimilar prescription targets increased the average share of biosimilars between 6 percentage points (P < .05) in Hamburg and up to 20 percentage points (P < .001) in Saxony-Anhalt. Stratification of specialists by prescription volume and adoption status indicated heterogeneous effects. We identified similar but higher effects for high-volume prescribers. Disentangling of effects with regard to the composition of biosimilar share suggested that the increase in biosimilar share was driven by increased biosimilar use accompanied by a nonsignificant decrease in original biologics prescriptions.

CONCLUSIONS

Prescription targets to alter physician prescribing behavior meet their intended goals by increasing biosimilar share. Physicians partly responded to the policy by decreasing overall prescriptions of the target substance. Prescription targets might be a useful tool, but decision makers need to consider all aspects of potential responses.

COVID-19, information management by local governments, and food consumption

Federal and state governments in developing countries have tasked local governments with managing COVID-19 on the ground. The bottom-up approach is critical to ensuring household food security, especially in rural areas. We have utilized data from a panel of Indian households that participated in two rounds of a livelihoods survey. While the first round was fielded before COVID-19, the second round was conducted telephonically after the COVID-19-lockdown. We developed an Information Management Response Index (IMRI) to measure the strength of local governments' information management initiatives. The difference-in-difference estimates show that local governments could partially mitigate the pandemic's adverse effects on (a) level and distribution (adult-equivalent per-capita) of food and nutrition expenditure and (b) household vulnerability to food and nutrition poverty. For landless households, IMRI led to statistically significant and additional welfare effects. Three channels explain our empirical findings: (a) maintenance of essential commodities through fair-price shops, (b) access to paid employment and cash (income effect), and (c) disease management (substitution effect). The estimates have been adjusted for sample attrition and multiple-hypothesis correction. We conducted robustness checks with respect to index construction, instrumental variable estimation, and sub-group analysis.

Performance bonuses and the quality of primary health care delivered by family health teams in Brazil: A difference-in-differences analysis

BACKGROUND

Pay-for-performance (P4P) programmes to incentivise health providers to improve quality of care have been widely implemented globally. Despite intuitive appeal, evidence on the effectiveness of P4P is mixed, potentially due to differences in how schemes are designed. We exploited municipality variation in the design features of Brazil's National Programme for Improving Primary Care Access and Quality (PMAQ) to examine whether performance bonuses given to family health team workers were associated with changes in the quality of care and whether the size of bonus mattered.

METHODS AND FINDINGS

For this quasi-experimental study, we used a difference-in-differences approach combined with matching. We compared changes over time in the quality of care delivered by family health teams between (bonus) municipalities that chose to use some or all of the PMAQ money to provide performance-related bonuses to team workers with (nonbonus) municipalities that invested the funds using traditional input-based budgets. The primary outcome was the PMAQ score, a quality of care index on a scale of 0 to 100, based on several hundred indicators (ranging from 598 to 660) of health care delivery. We did one-to-one matching of bonus municipalities to nonbonus municipalities based on baseline demographic and economic characteristics. On the matched sample, we used ordinary least squares regression to estimate the association of any bonus and size of bonus with the prepost change over time (between November 2011 and October 2015) in the PMAQ score. We performed subgroup analyses with respect to the local area income of the family health team. The matched analytical sample comprised 2,346 municipalities (1,173 nonbonus municipalities; 1,173 bonus municipalities), containing 10,275 family health teams that participated in PMAQ from the outset. Bonus municipalities were associated with a 4.6 (95% CI: 2.7 to 6.4; p < 0.001) percentage point increase in the PMAQ score compared with nonbonus municipalities. The association with quality of care increased with the size of bonus: the largest bonus group saw an improvement of 8.2 percentage points (95% CI: 6.2 to 10.2; p < 0.001) compared with the control. The subgroup analysis showed that the observed improvement in performance was most pronounced in the poorest two-fifths of localities. The limitations of the study include the potential for bias from unmeasured time-varying confounding and the fact that the PMAQ score has not been validated as a measure of quality of care.

CONCLUSIONS

Performance bonuses to family health team workers compared with traditional input-based budgets were associated with an improvement in the quality of care.

Association of Rural Hospital Admissions with Access, Treatment, and Mortality for Patients with Acute Myocardial Infarction in Shanxi, China

China recently launched healthcare reforms to reduce disparities in healthcare resources between urban and rural areas. However, few studies have determined how admission to rural hospitals has affected patient care and outcomes. This study aims to determine whether admission to a rural hospital is associated with changes in treatment and outcomes. Using a province-wide, administrative database of 62,380 patients (51,355 urban patients vs. 11,025 rural patients) with acute myocardial infarction (AMI) in Shanxi from 2015 to 2017, we identified the differential distance from the patient’s residential address to the nearest hospital and the nearest percutaneous coronary intervention (PCI)-capable hospital as instrumental variables. We estimated the risk-adjusted differences in outcomes and treatments for patients admitted to rural hospitals versus urban hospitals using a two-stage least squares instrumental variable analysis method. Based on instrumental variable analysis, admission to a rural hospital was associated with a 5.3% (95% CI, 0.012 to 0.093; p = 0.011) increase in mortality. There was a 59.8% (95% CI, −0.733 to −0.463; p-values < 0.0001) decrease in receiving PCI, an 18.8% (95% CI, −0.231 to −0.146; p-values < 0.0001) decrease in receiving fibrinolysis, and a 71.8% (95% CI, 0.586 to 0.849; p-values < 0.0001) increase in receiving medication-only treatment for patients admitted to rural hospitals. Rural hospitals in China thus offer relatively poor care for myocardial infarction. Hospital facilities and reperfusion therapies must be improved.

Evaluation of the mental health impacts of Universal Credit: protocol for a mixed methods study

INTRODUCTION

The UK social security system is being transformed by the implementation of Universal Credit (UC), which combines six existing benefits and tax credits into a single payment for low-income households. Despite extensive reports of hardship associated with the introduction of UC, no previous studies have comprehensively evaluated its impact on mental health. Because payments are targeted at low-income households, impacts on mental health will have important consequences for health inequalities.

METHODS AND ANALYSIS

We will conduct a mixed methods study. Work package (WP) 1 will compare health outcomes for new recipients of UC with outcomes for legacy benefit recipients in two large population surveys, using the phased rollout of UC as a natural experiment. We will also analyse the relationship between the proportion of UC claimants in small areas and a composite measure of mental health. WP2 will use data collected by Citizen's Advice to explore the sociodemographic and health characteristics of people who seek advice when claiming UC and identify features of the claim process that prompt advice-seeking. WP3 will conduct longitudinal in-depth interviews with up to 80 UC claimants in England and Scotland to explore reasons for claiming and experiences of the claim process. Up to 30 staff supporting claimants will also be interviewed. WP4 will use a dynamic microsimulation model to simulate the long-term health impacts of different implementation scenarios. WP5 will undertake cost-consequence analysis of the potential costs and outcomes of introducing UC and cost-benefit analyses of mitigating actions.

ETHICS AND DISSEMINATION

We obtained ethical approval for the primary data gathering from the University of Glasgow, College of Social Sciences Research Ethics Committee, application number 400200244. We will use our networks to actively disseminate findings to UC claimants, the public, practitioners and policy-makers, using a range of methods and formats.

TRIAL REGISTRATION NUMBER

The study is registered with the Research Registry: researchregistry6697.

The Influence of Loneliness on the Smoking and Physical Activity of Community-Dwelling Older Adults: Results from the Health and Retirement Study

PURPOSE

To use the loneliness model in examining the influence of loneliness on the number cigarettes smoked per day and the different intensity levels of physical activity among community-dwelling older Americans in the United States.

DESIGN, SETTING, SAMPLE

This study analyzed a nationally representative sample of older adults aged 65+ in two waves (2010 and 2012) of data from the Health and Retirement Study. Response rates for the two waves were 81% and 89.1%. The sample size for smoking model was 199, and for physical activity models was 3018.

MEASURES

Outcomes included number of cigarettes smoked per day and physical activity at three intensity levels: light, moderate, and vigorous. Independent variable was the UCLA loneliness scale.

ANALYSIS

A lagged dependent approach for modeling longitudinal data was adopted. Models controlled for outcomes at the first timepoint (Wave 1), health/physical functioning, and demographic variables.

RESULTS

Loneliness was associated with an increased number of cigarettes smoked per day (β = 2.93, P < .05) and decreased engagement in moderate and vigorous physical activity for older adults (β = .12, P < .05; β = .12, P <. 05), after controlling for these behaviors at baseline and other covariates.

CONCLUSION

The findings call for smoking reduction and physical activity enhancement interventions targeting older adults who have high levels of loneliness. Efforts to enhance social support will be crucial to eradicating the harmful health impact of loneliness. Critical limitations include self-reported measures and unobserved confounders.

Using Negative Control Outcomes and Difference-in-Differences Analysis to Estimate Treatment Effects in an Entirely Treated Cohort: The Effect of Ivacaftor in Cystic Fibrosis

When an entire cohort of patients receives a treatment, it is difficult to estimate the treatment effect in the treated because there are no directly comparable untreated patients. Attempts can be made to find a suitable control group (e.g., historical controls), but underlying differences between the treated and untreated can result in bias. Here we show how negative control outcomes combined with difference-in-differences analysis can be used to assess bias in treatment effect estimates and obtain unbiased estimates under certain assumptions. Causal diagrams and potential outcomes are used to explain the methods and assumptions. We apply the methods to UK Cystic Fibrosis Registry data to investigate the effect of ivacaftor, introduced in 2012 for a subset of the cystic fibrosis population with a particular genotype, on lung function and annual rate (days/year) of receiving intravenous (IV) antibiotics (i.e., IV days). We consider 2 negative control outcomes: outcomes measured in the pre-ivacaftor period and outcomes among persons ineligible for ivacaftor because of their genotype. Ivacaftor was found to improve lung function in year 1 (an approximately 6.5–percentage-point increase in ppFEV₁), was associated with reduced lung function decline (an approximately 0.5–percentage-point decrease in annual ppFEV₁ decline, though confidence intervals included 0), and reduced the annual rate of IV days (approximately 60% over 3 years).

Moving beyond the classic difference-in-differences model: a simulation study comparing statistical methods for estimating effectiveness of state-level policies

Background

Reliable evaluations of state-level policies are essential for identifying effective policies and informing policymakers’ decisions. State-level policy evaluations commonly use a difference-in-differences (DID) study design; yet within this framework, statistical model specification varies notably across studies. More guidance is needed about which set of statistical models perform best when estimating how state-level policies affect outcomes.

Methods

Motivated by applied state-level opioid policy evaluations, we implemented an extensive simulation study to compare the statistical performance of multiple variations of the two-way fixed effect models traditionally used for DID under a range of simulation conditions. We also explored the performance of autoregressive (AR) and GEE models. We simulated policy effects on annual state-level opioid mortality rates and assessed statistical performance using various metrics, including directional bias, magnitude bias, and root mean squared error. We also reported Type I error rates and the rate of correctly rejecting the null hypothesis (e.g., power), given the prevalence of frequentist null hypothesis significance testing in the applied literature.

Results

Most linear models resulted in minimal bias. However, non-linear models and population-weighted versions of classic linear two-way fixed effect and linear GEE models yielded considerable bias (60 to 160%). Further, root mean square error was minimized by linear AR models when we examined crude mortality rates and by negative binomial models when we examined raw death counts. In the context of frequentist hypothesis testing, many models yielded high Type I error rates and very low rates of correctly rejecting the null hypothesis (< 10%), raising concerns of spurious conclusions about policy effectiveness in the opioid literature. When considering performance across models, the linear AR models were optimal in terms of directional bias, root mean squared error, Type I error, and correct rejection rates.

Conclusions

The findings highlight notable limitations of commonly used statistical models for DID designs, which are widely used in opioid policy studies and in state policy evaluations more broadly. In contrast, the optimal model we identified--the AR model--is rarely used in state policy evaluation. We urge applied researchers to move beyond the classic DID paradigm and adopt use of AR models.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01471-y.

Effects of Patent Policy on Outputs and Commercialization of Academic Patents in China: A Spatial Difference-in-Differences Analysis

The development of a difference-in-differences estimator is a new move in patent policy evaluation research. However, such an estimator neglects the possibility that academic patent activities follow a spatial autoregressive process with respect to the dependent variable. The objective of this study was to propose a spatial difference-in-differences estimator accounting for possible spatial spillover effects. In this study, an empirical analysis of a sample of 31 Chinese provinces from 2010 to 2019 indicates that an incentive patent policy has a positive impact on the output and commercialization of academic patents, with positive effects also spilling over into neighboring provinces. This study further found that incentive patent policies play a placebo role in academic patent activities. Provincial patent policies are merely a proxy for other variables that characterize the systemic differences between provinces that implement patent policies and those that do not. Therefore, the promotion of academic patent activities cannot be attributed to policy incentives.

Post robotic investment: Cost consequences and impact on length of stay for obese women with endometrial cancer

INTRODUCTION

The aim of the study was to investigate whether robotic-assisted surgery is associated with lower incremental resource use among obese patients relative to non-obese patients after a Danish nationwide adoption of robotic-assisted surgery in women with early-stage endometrial cancer. This is a population-based cohort study based on registers and clinical data.

MATERIAL AND METHODS

All women who underwent surgery (robotic, laparoscopic and laparotomy) from 2008 to 2015 were included and divided according to body mass index (<30 and ≥30). Robotic-assisted surgery was gradually introduced in Denmark (2008-2013). We compared resource use post-surgery in obese vs non-obese women who underwent surgery before and after a nationwide adoption of robotic-assisted surgery. The key exposure variable was exposure to robotic-assisted surgery. Clinical and sociodemographic data were linked with national register data to determine costs and bed days 12 months before and after surgery applying difference-in-difference analyses.

RESULTS

In total, 3934 women were included. The adoption of robotic-assisted surgery did not demonstrate statistically significant implications for total costs among obese women (€3,417; 95% confidence interval [CI] -€854 to €7,688, p = 0.117). Further, for obese women, a statistically significant reduction in bed days related to the index hospitalization was demonstrated (-1.9 bed days; 95% CI -3.6 to -0.2, p = 0.025). However, for non-obese women, the adoption of robotic-assisted surgery was associated with statistically significant total costs increments of €9,333 (95% CI €3,729-€1,4936, p = 0.001) and no reduction in bed days related to the index hospitalization was observed (+0.9 bed days; 95% CI -0.6 to 2.3, p = 0.242).

CONCLUSIONS

The national investment in robotic-assisted surgery for endometrial cancer seems to have more modest cost implications post-surgery for obese women. This may be partly driven by a significant reduction in bed days related to the index hospitalization among obese women, as well as reductions in subsequent hospitalizations.

Does coal-to-gas policy reduce air pollution? Evidence from a quasi-natural experiment in China

Whether the use of cleaner energy can reduce air pollution is the focus of debate among scholars, but there is still no unanimous conclusion. This study seeks to explore the net impact of coal-to-gas policy, an energy transition policy in China, on air pollution. Utilizing prefecture-level city data from 2003 to 2016, we apply the PSM-DID method to estimate the policy's net impact. Further, we examine the dynamic effects of coal-to-gas policy and its impact mechanism on air pollution. The results show that 1) The coal-to-gas policy has an average reduction effect of 31.3%, 36%, 0.3%, and 33.1% on industrial sulfur dioxide (SO₂), industrial Smoke (dust), fine particulate matter (PM_2.5), and air quality index (AQI). After eliminating the spreading interference of PM_2.5 in surrounding areas, the effect of this policy on PM_2.5 reduction is 7%; 2) the impact of the coal-to-gas policy is significant in 2012 and 2013, i.e. the second and third years after the implementation of the policy. Then, the reduction effect of the policy on air pollution began to decrease; 3) the coal-to-gas policy has led to the increase in the proportion of the tertiary industry and the decrease in the degree of industrialization. Since the development of the tertiary industry and the reduction of industrialization also led to a reduction in air pollution, the coal-to-gas policy can reduce air pollution through industrial structure upgrading and de-industrialization. The robustness test results support the above conclusions. Practicable policies to reduce air pollution in China are suggested and applicable to other developing countries with resource-scarce and serve air pollution.

Analytical methods to assess the impacts of activity-based funding (ABF): a scoping review

BACKGROUND

Activity-Based Funding (ABF) has been implemented across many countries as a means to incentivise efficient hospital care delivery and resource use. Previous reviews have assessed the impact of ABF implementation on a range of outcomes across health systems. However, no comprehensive review of the methods used to generate this evidence has been undertaken. The aim of this review is to identify and assess the analytical methods employed in research on ABF hospital performance outcomes.

METHODS

We conducted a scoping review in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews. Five academic databases and reference lists of included studies were used to identify studies assessing the impact of ABF on hospital performance outcomes. Peer-reviewed quantitative studies published between 2000 and 2019 considering ABF implementation outside the U.S. were included. Qualitative studies, policy discussions and commentaries were excluded. Abstracts and full text studies were double screened to ensure consistency. All analytical approaches and their relative strengths and weaknesses were charted and summarised.

RESULTS

We identified 19 studies that assessed hospital performance outcomes from introduction of ABF in England, Korea, Norway, Portugal, Israel, the Netherlands, Canada, Italy, Japan, Belgium, China, and Austria. Quasi-experimental methods were used across most reviewed studies. The most commonly used assessment methods were different forms of interrupted time series analyses. Few studies used difference-in-differences or similar methods to compare outcome changes over time relative to comparator groups. The main hospital performance outcome measures examined were case numbers, length of stay, mortality and readmission.

CONCLUSIONS

Non-experimental study designs continue to be the most widely used method in the assessment of ABF impacts. Quasi-experimental approaches examining the impact of ABF implementation on outcomes relative to comparator groups not subject to the reform should be applied where possible to facilitate identification of effects. These approaches provide a more robust evidence-base for informing future financing reform and policy.

Confounding and regression adjustment in difference-in-differences studies

Objective

To define confounding bias in difference‐in‐difference studies and compare regression‐ and matching‐based estimators designed to correct bias due to observed confounders.

Data sources

We simulated data from linear models that incorporated different confounding relationships: time‐invariant covariates with a time‐varying effect on the outcome, time‐varying covariates with a constant effect on the outcome, and time‐varying covariates with a time‐varying effect on the outcome. We considered a simple setting that is common in the applied literature: treatment is introduced at a single time point and there is no unobserved treatment effect heterogeneity.

Study design

We compared the bias and root mean squared error of treatment effect estimates from six model specifications, including simple linear regression models and matching techniques.

Data collection

Simulation code is provided for replication.

Principal findings

Confounders in difference‐in‐differences are covariates that change differently over time in the treated and comparison group or have a time‐varying effect on the outcome. When such a confounding variable is measured, appropriately adjusting for this confounder (ie, including the confounder in a regression model that is consistent with the causal model) can provide unbiased estimates with optimal SE. However, when a time‐varying confounder is affected by treatment, recovering an unbiased causal effect using difference‐in‐differences is difficult.

Conclusions

Confounding in difference‐in‐differences is more complicated than in cross‐sectional settings, from which techniques and intuition to address observed confounding cannot be imported wholesale. Instead, analysts should begin by postulating a causal model that relates covariates, both time‐varying and those with time‐varying effects on the outcome, to treatment. This causal model will then guide the specification of an appropriate analytical model (eg, using regression or matching) that can produce unbiased treatment effect estimates. We emphasize the importance of thoughtful incorporation of covariates to address confounding bias in difference‐in‐difference studies.

How does reorganisation in child and adolescent mental health services affect access to services? An observational study of two services in England

BACKGROUND

Child and Adolescent Mental Health Services (CAMHS) in England are making significant changes to improve access and effectiveness. This 'transformation' variously involves easier access to services through a Single Point of Access (SPA), more integrated services within CAMHS and enhanced co-provision across education and third sector or non-profit organisations.

METHODS

A mixed-methods observational study was conducted to explore the process and impact of transformation over four years in two services. Ethnographic observations and in-depth interviews were conducted and Electronic Patient Records with over one million contacts analysed. Difference-in-differences analysis with propensity score matching to estimate the causal impact of the transformation on patient access was utilised.

OUTCOMES

Spend and staffing increased across both CAMHS. The SPA had growing rates of self-referral and new care pathways were seeing patients according to expected degree of psychopathology. Third sector partners were providing increasing numbers of low-intensity interventions. Although the majority of staff were supportive of the changes, the process of transformation led to service tensions. In the first year after transformation there was no change in the rate of new patients accessing services or new spells (episodes of care) in the services. However, by year three, the number of new patients accessing CAMHS was 19% higher (Incidence Rate Ratio: 1·19, CI: 1·16, 1·21) and the rate of new spells was 12% higher (Incidence Rate Ratio: 1·12, CI: 1·05, 1·20).

INTERPRETATION

Transformation investment, both financial and intellectual, can help to increase access to CAMHS in England, but time is needed to realise the benefits of reorganisation.

Containing costs in the Italian local healthcare market

We aim to investigate the cost containment effects of the creation in 2005 of agencies specifically responsible for all technical and administrative services within the regional healthcare system of the Tuscany region of Italy. We seek to contribute to the existing literature on the centralization and decentralization of purchases and technical services by assessing the amount of savings produced by these agencies at the intermediate level between local authorities and hospitals and the regional administration. We use the balances of all Italian local health authorities and hospital trusts combined with the synthetic control procedure to create from a donor pool of untreated units a weighted average of observations resembling the exposed units before and after the policy change in 2005. The magnitude of the effect is significant as the creation of these agencies is estimated to have reduced expenditures on auxiliary goods and contracted services by 6% in the period from 2006-2014. Moreover, we find that the cost reduction is not associated with a decrease in the provision of healthcare services and procedures to the general population or in the quality or efficiency of the regional healthcare system itself.

Impact of social accountability monitoring on health facility performance: Evidence from Tanzania

Social accountability programs are increasingly used to improve the performance of public service providers in low-income settings. Despite their growing popularity, evidence on the effectiveness of social accountability programs remains mixed. In this manuscript, we assess the impact of a social accountability intervention on health facility management exploring quasiexperimental variation in program exposure in Tanzania. We find that the social accountability intervention resulted in a 1.8 SD reduction in drug stockouts relative to the control group, but did not improve facility infrastructure maintenance. The results of this study suggest that social accountability programs may be effective in areas of health service provision that are responsive to changes in provider behavior but may not work in settings where improvements in outcomes are conditional on larger health systems features.

Impact of Electronic Prescribing of Controlled Substances on Opioid Prescribing: Evidence From I-STOP Program in New York

New York's Internet System for Tracking Over-Prescribing (I-STOP) Act, requires prescribers in the state to electronically prescribe controlled substances (EPCS). We examine the effects of this mandate on prescribing patterns of opioids for Medicare Part D beneficiaries. Using 2014-2017 CMS Medicare Part D Prescriber Data, we apply a lagged dependent variable regression approach to identify the impact of I-STOP on the prescription of opioids. In the first year of implementation, the number of opioid prescriptions per prescriber decreased by 5.7 per year. The policy had a larger effect on the prescription of short-acting opioids and on prescribers prescribing medication for predominantly younger beneficiaries. Overall, I-STOP resulted in a reduction in the number of beneficiaries being prescribed opioids and in the number of opioid claims in the state of New York, suggesting positive implications for other states intending to curtail opioid overprescribing and misuse through the use of EPCS.

Can General Practitioners Be More Productive? The Impact of Teamwork and Cooperation with Nurses on GP Activities

The integration of primary care organizations and interprofessional cooperation is encouraged in many countries to both improve the productive and allocative efficiency of care provision and address the unequal geographical distribution of general practitioners (GPs). In France, a pilot experiment promoted the vertical integration of and teamwork between GPs and nurses. This pilot experiment relied on the staffing and training of nurses; skill mixing, including the authorization to shift tasks from GPs to nurses; and new remuneration schemes. This article evaluates the overall impact of this pilot experiment over the period 2010-2017 on GP activities based on the following indicators: number of working days, patients seen at least once, patients registered, and visits delivered. We control for endogeneity and reduce selection bias by using a case-control design combining coarsened exact matching and difference-in-differences estimates on panel data. We find a small positive impact on the number of GP working days (+1.2%) following enrollment and a more pronounced effect on the number of patients seen (+7.55%) or registered (+6.87%). However, we find no effect on the number of office and home visits. In this context, cooperation and teamwork between GPs and nurses seem to improve access to care for patients.

Evaluating the power of the causal impact method in observational studies of HCV treatment as prevention

OBJECTIVES

The causal impact method (CIM) was recently introduced for evaluation of binary interventions using observational time-series data. The CIM is appealing for practical use as it can adjust for temporal trends and account for the potential of unobserved confounding. However, the method was initially developed for applications involving large datasets and hence its potential in small epidemiological studies is still unclear. Further, the effects that measurement error can have on the performance of the CIM have not been studied yet. The objective of this work is to investigate both of these open problems.

METHODS

Motivated by an existing dataset of HCV surveillance in the UK, we perform simulation experiments to investigate the effect of several characteristics of the data on the performance of the CIM. Further, we quantify the effects of measurement error on the performance of the CIM and extend the method to deal with this problem.

RESULTS

We identify multiple characteristics of the data that affect the ability of the CIM to detect an intervention effect including the length of time-series, the variability of the outcome and the degree of correlation between the outcome of the treated unit and the outcomes of controls. We show that measurement error can introduce biases in the estimated intervention effects and heavily reduce the power of the CIM. Using an extended CIM, some of these adverse effects can be mitigated.

CONCLUSIONS

The CIM can provide satisfactory power in public health interventions. The method may provide misleading results in the presence of measurement error.

The impact of introducing a national scheme for paid parental leave on maternal mental health outcomes

Paid maternity leave is an essential component of a progressive society. It can enhance postnatal health, improve mother and child wellbeing, and deliver better labor market outcomes for mothers. We evaluate the impact of the introduction of Australia's national Paid Parental Leave (PPL) scheme in 2011 and complementary Dad and Partner Pay (DAPP) in 2013 on maternal mental health. Using a sample of 1480 births to eligible, partnered women between 2004 and 2016 and examining a range of mental health outcomes from the Household, Income, andLabour Dynamics in Australia survey, we find depression likelihood reduced significantly in post-reform years. When focusing on post-DAPP years and women whose partners had concurrent access to DAPP, significant mental health improvements were found across a wider range of measures including the Mental Component Summary score and specific Short Form-36 items with a high sensitivity for detecting major depression. Subgroup analysis suggests significant improvements applied specifically to first-time mothers and mothers with employer-paid maternity leave and unpaid leave entitlements. These results suggest that an increase in PPL and DAPP entitlements for mothers without access to employer-paid and unpaid leave entitlements, particularly those in less secure employment, may further reduce postnatal depression and improve health equity in Australia.

Impact of Brazil's More Doctors Program on hospitalizations for primary care sensitive cardiovascular conditions

Globally, cardiovascular diseases are the leading cause of disease burden and death. Timely and appropriate provision of primary care may lead to sizeable reductions in hospitalizations for a range of chronic and acute health conditions. In this paper, we study the impact of Brazil's More Doctors Program (MDP) on hospitalizations due to cerebrovascular disease and hypertension. We exploit the geographic variation in the uptake of the MPD and combine coarsened exact matching and difference-in-difference methods to construct valid counterfactual estimates. We use data from the Hospital Information System in Unified Health System, the MDP administrative records, the Brazilian Regulatory Agency, the Ministry of Health, and the Brazilian Institute of Geography and Statistics, covering the years from 2009 to 2017. Our analysis resulted in estimated coefficients of -1.47 (95%CI: -4.04,1.10) for hospitalizations for cerebrovascular disease and -1.20 (95%CI: -5.50,3.11) for hypertension, suggesting an inverse relationship between the MDP and hospitalizations. For cerebrovascular disease, the estimated MDP coefficient was -0.50 (95%CI: -2.94,1.95) in the year of program introduction, -5.21 (95%CI: -9.43,-0.99) and -8.21 (95%CI: -13.68,-2.75) in its third and fourth year of implementation, respectively. Our results further suggest that the beneficial impact of MDP on hospitalizations due to cerebrovascular disease became discernable in urban municipalities starting from the fourth year of implementation. We found no evidence that the MDP led to reductions in hospitalizations due to hypertension. Our results highlight that increased investment in resources devoted to primary care led to improvements in hospitalizations for selected cardiovascular conditions. However, it took time for the beneficial effects of the MDP to become discernable and the Program did not guarantee declines in hospitalizations for all cardiovascular conditions, suggesting that further improvements may be needed to enhance the beneficial impact of the MDP on the level and distribution of population health in Brazil.

Is 'minimally adequate treatment' really adequate? investigating the effect of mental health treatment on quality of life for children with mental health problems

BACKGROUND

Minimally adequate treatment (MAT) is intended to represent treatment minimally sufficient for common mental health problems. For children, MAT has been defined over a twelve-month period as either eight or more mental health visits, or four to seven visits plus relevant medication. MAT is used to identify those missing out on adequate care, but it is unknown whether MAT improves children's outcomes.

METHODS

This paper examines whether MAT is associated with improved outcomes for children. It uses survey data from the nationally representative Longitudinal Study of Australian children on 596 children with mental health problems based on the Strengths and Difficulties Questionnaire at ages 8-15 years, linked to health service administrative data from 2012 to 2016. Statistical analysis examines the association of MAT with later quality of life (Pediatric Quality of Life Inventory), using a lagged dependent variable model to account for time-varying unobserved confounding.

RESULTS

Compared to children with lower levels of treatment, those who received MAT between baseline and follow up had no statistically significant improvement in either quality of life or mental health symptoms.

LIMITATIONS

The observational data provide insight into real-world practice but require statistical methods to account for selection into treatment.

CONCLUSIONS

While clinical trials show mental health treatments can be efficacious, this study shows no evidence that children receiving MAT in routine practice have better outcomes. These findings demonstrate the need for better understanding of the nature and impact of children's mental health care as it is delivered and received in routine practice.

Effects of Financial Inclusion on Access to Emergency Funds for Healthcare in the Kingdom of Saudi Arabia

Background

Having access to convenient and quality healthcare at all times is not only a human right but also a goal that many countries strive to achieve for their population. However, access to healthcare might face blocks in the presence of financial exclusions. Saudi Arabia has, over the years, continued to pursue policy and system reforms to enhance its population’s access to financial inclusion, as well as proper health coverage to improve health outcomes. This study seeks to estimate the effects of financial inclusion on the financial hardships in accessing healthcare in Saudi Arabia.

Methods

This study uses a nationally representative survey conducted with 1009 adults, using the 2017 World Bank Global Findex Study data. The study estimates the conditional probability of coming up with emergency funds and the conditional probability of borrowing for medical purposes to understand access to healthcare. A composite value is created for financial inclusion using several variables for individuals’ interactions with financial institutions, such as access to financial services and loans.

Results

The results revealed that financially included individuals have a higher conditional probability of both borrowing for medical purposes and coming up with emergency funds, compared to those who are financially excluded. Additionally, the study showed that individuals in low-income brackets are more likely to be financially excluded and have a reduced chance of coming up with emergency funds and borrowing for medical purposes.

Conclusion

These findings indicate that there is need for authorities to roll out a financial inclusion drive that will not only incentivise the financially excluded population to become included but that will also aim at promoting various financial products so that those who are already financially included have a wide range from which they can choose.

Income support, employment transitions and well-being

Using specific panel data of German welfare benefit recipients, we investigate the non-pecuniary life satisfaction effects of in-work benefits. Our empirical strategy combines difference-in-difference designs with synthetic control groups to analyse transitions of workers between unemployment, regular employment and employment accompanied by welfare receipt. Working makes people generally better off than being unemployed but employed welfare recipients do not reach the life satisfaction level of regular employees. This implies that welfare receipt entails non-compliance with the norm to make one's own living. Our findings allow us to draw cautious conclusions on employment subsidies paid as welfare benefits.

Evaluation of an Implementation Intervention to Increase Reach of Evidence-Based Psychotherapies for PTSD in US Veterans Health Administration PTSD Clinics

To evaluate an implementation intervention to increase the uptake, referred to as reach, of two evidence-based psychotherapies (EBP) for posttraumatic stress disorder (PTSD) in Veterans Health Administration (VHA) PTSD specialty clinics. The implementation intervention was external facilitation guided by a toolkit that bundled strategies associated with high EBP reach in prior research. We used a prospective quasi-experimental design. The facilitator worked with local champions at two low-reach PTSD clinics. Each intervention PTSD clinic was matched to three control clinics. We compared the change in EBP reach from 6-months pre- to post-intervention using Difference-in-Difference (DID) effect estimation. To incorporate possible clustering effects and adjust for imbalanced covariates, we used mixed effects logistic regression to model the probability of EBP receipt. Analyses were conducted separately for PTSD and other mental health clinics. 29,446 veterans diagnosed with PTSD received psychotherapy in the two intervention and six control sites in the two 6-month evaluation periods. The proportion of therapy patients with PTSD receiving an EBP increased by 16.98 percentage points in the intervention PTSD clinics compared with .45 percentage points in the control PTSD clinics (DID = 16.53%; SE = 2.26%). The adjusted odd ratio of a patient receiving an EBP from pre to post intervention was almost three times larger in the intervention than in the control PTSD clinics (RoR 2.90; 95% CI 2.22-3.80). EBP reach was largely unchanged in other (not PTSD specialty) mental health clinics within the same medical centers. Toolkit-guided external facilitation is a promising intervention to improve uptake of EBPs in VHA. Toolkits that pre-specify targets for clinic change based on prior research may enhance the efficiency and effectiveness of external facilitation. Trial registration ISRCTN registry identifier: ISRCTN65119065. Available at https://www.isrctn.com/search?q=ISRCTN65119065 .

Effects of Point-Of-Care Testing in General Practice for Type 2 Diabetes Patients on Ambulatory Visits and Hospitalizations

Point-of-care testing (POCT) of HbA1c means instant test results and more coherent counseling that is expected to improve diabetes management and affect ambulatory visits and hospitalizations. From late 2008, POCT has been implemented and adopted by a segment of the general practices in the capital region of Denmark. The aim of this study is to assess whether the introduction of POCT of HbA1c in general practice (GP) has affected patient outcomes for T2 diabetes patients in terms of hospital activity. We apply difference-in-differences models at the GP clinic level to assess the casual effects of POCT on the following hospital outcomes: (1) admissions for diabetes, (2) admissions for ambulatory care sensitive diabetes conditions (ACSCs), (3) ambulatory visits for diabetes. The use of POCT is remunerated by a fee, and registration of this fee is used to measure the GP’s use of POCT. The control group includes clinics from the same region that did not use POCT. The sensitivity of our results is assessed by an event study approach and a range of robustness tests. The panel data set includes 553 GP clinics and approximately 30,000 diabetes patients from the capital region of Denmark, observed in the years 2004–2012. We find that voluntary adoption of POCT of HbA1c in GP has no effect on hospital admissions and diabetes-related hospital ambulatory visits. Event study analysis and different treatment definitions confirm the robustness of these results. If implementation of POCT of HbA1c improves other parts of diabetes management as indicated in the literature, it seems worthwhile to implement POCT of HbA1c in the capital region of Denmark. However, doubts around the quality of POCT of HbA1c testing and a desire to capture data at central labs may prevent implementation of more value based HbA1c testing.

Can a new emergency response system reduce traffic fatalities? The case of the 911-emergency response system in the Dominican Republic

BACKGROUND

In May 2014, the Dominican Republic introduced the 911-emergency response system (ERS) in Santo Domingo. Before its introduction, more than 40 phone numbers were available to report emergencies. The objective of this work is to assess whether this new emergency response system was effective in reducing traffic fatalities.

METHODS

Weekly numbers of traffic fatalities per population and per vehicle fleet from January 2013 to December 2015 were obtained from the Ministry of Health and the National Institute of Statistics. A hybrid time-series difference-in-difference analysis using multivariable negative binomial regression models were used to compare trends in rates of traffic fatalities in Santo Domingo to La Romana and Santiago, before and after the introduction of the 911-ERS.

RESULTS

Estimates from negative binomial models suggest that the introduction of the 911-ERS in Santo Domingo relative to Santiago-La Romana was associated with a 17% reduction in the Incidence Rate Ratio (IRR) of traffic fatalities per 1 000 000 population (IRR = 0.83, 95% confidence interval [CI]: 0.67; 1.03) and with a 20% reduction in the IRR of weekly traffic fatalities per 1 000 000 vehicle fleet (IRR = 0.80, 95% CI:0.67; 0.99).

DISCUSSION

Our findings suggest that transitioning from multiple to one unique emergency phone number should be considered more attentively. Furthermore, the case of the Dominican Republic calls for more theoretical and methodological research to understand how to assess these road safety policies more accurately. Since various studies suggest that 911-ERS mature in the long run, how these systems evolve over time and other related variables should be carefully considered.

Measuring the impact of national guidelines: What methods can be used to uncover time-varying effects for healthcare evaluations?

We examine the suitability of three methods using patient-level data to evaluate the time-varying impacts of national healthcare guidelines. Such guidelines often codify progressive change and are implemented gradually; for example, National Institute for Health and Care Excellence (NICE) suspected-cancer referral guidelines. These were revised on June 23, 2015, to include more cancer symptoms and test results ("features"), partly reflecting changing practice. We explore the time-varying impact of guideline revision on time to colorectal cancer diagnosis, which is linked to improved outcomes in decision-analytic models. We included 11,842 patients diagnosed in 01/01/2006-31/12/2017 in the Clinical Practice Research Datalink with England cancer registry data linkage. Patients were classified by whether their first pre-diagnostic cancer feature was in the original guidelines (NICE-2005) or was added during the revision (NICE-2015-only). Outcome was diagnostic interval: time from first cancer feature to diagnosis. All analyses adjusted for age and sex. Two difference-in-differences analyses used either a Pre (01/08/2012-31/12/2014, n = 2243) and Post (01/08/2015-31/12/2017, n = 1017) design, or event-study cohorts (2006-2017 vs 2015) to estimate change in diagnostic interval attributable to official implementation of the revised guidelines. A semiparametric varying-coefficient model analysed the difference in diagnostic interval between the NICE groups over time. After model estimation, primary and broader treatment effects of guideline content and implementation were measured. The event-study difference-in-differences and the semiparametric varying-coefficient methods showed that shorter diagnostic intervals were attributable to official implementation of the revised guidelines. This impact was only detectable by pre-to-post difference-in-differences when the pre/post periods were selected according to the estimation results from the varying-coefficient model. Formal tests of the parametric models, which are special cases of the semiparametric model, suggest that they are misspecified. We conclude that the semiparametric method is well suited to explore the time-varying impacts of guidelines codifying progressive change.

A novel method for controlling unobserved confounding using double confounders

BACKGROUND

Controlling unobserved confounding still remains a great challenge in observational studies, and a series of strict assumptions of the existing methods usually may be violated in practice. Therefore, it is urgent to put forward a novel method.

METHODS

We are interested in the causal effect of an exposure on the outcome, which is always confounded by unobserved confounding. We show that, the causal effect of an exposure on a continuous or categorical outcome is nonparametrically identified through only two independent or correlated available confounders satisfying a non-linear condition on the exposure. Asymptotic theory and variance estimators are developed for each case. We also discuss an extension for more than two binary confounders.

RESULTS

The simulations show better estimation performance by our approach in contrast to the traditional regression approach adjusting for observed confounders. A real application is separately applied to assess the effects of Body Mass Index (BMI) on Systolic Blood Pressure (SBP), Diastolic Blood Pressure (DBP), Fasting Blood Glucose (FBG), Triglyceride (TG), Total Cholesterol (TC), High Density Lipoprotein (HDL) and Low Density Lipoprotein (LDL) with individuals in Shandong Province, China. Our results suggest that SBP increased 1.60 (95% CI: 0.99-2.93) mmol/L with per 1- kg/m2 higher BMI and DBP increased 0.37 (95% CI: 0.03-0.76) mmol/L with per 1- kg/m2 higher BMI. Moreover, 1- kg/m2 increase in BMI was causally associated with a 1.61 (95% CI: 0.96-2.97) mmol/L increase in TC, a 1.66 (95% CI: 0.91-55.30) mmol/L increase in TG and a 2.01 (95% CI: 1.09-4.31) mmol/L increase in LDL. However, BMI was not causally associated with HDL with effect value - 0.20 (95% CI: - 1.71-1.44). And, the effect value of FBG per 1- kg/m2 higher BMI was 0.56 (95% CI: - 0.24-2.18).

CONCLUSIONS

We propose a novel method to control unobserved confounders through double binary confounders satisfying a non-linear condition on the exposure which is easy to access.

Cigarette Pack Prices and Sales Following Policy Changes in California, 2011-2018

Objectives. To estimate the combined effect of California's Tobacco 21 law (enacted June 2016) and $2-per-pack cigarette excise tax increase (enacted April 2017) on cigarette prices and sales, compared with matched comparator states.Methods. We used synthetic control methods to compare cigarette prices and sales after the policies were enacted, relative to what we would have expected without the policy reforms. To estimate the counterfactual, we matched pre-reform covariate and outcome trends between California and control states to construct a "synthetic" California.Results. Compared with the synthetic control in 2018, cigarette prices in California were $1.89 higher ($7.86 vs $5.97; P < .001), and cigarette sales were 16.6% lower (19.9 vs 16.6 packs per capita; P < .001). This reduction in sales equates to 153.9 million fewer packs being sold between 2017 and 2018.Conclusions. California's new cigarette tax was largely passed on to consumers. The new cigarette tax, combined with the Tobacco 21 law, have contributed to a rapid and substantial reduction in cigarette consumption in California.

Cross-Platform Spillover Effects in Consumption of Viral Content: A Quasi-Experimental Analysis Using Synthetic Controls

To inform product release and distribution strategies, research has analyzed cross-market spillovers in new product adoption. However, models that examine these effects for digital and viral media are still evolving. Given resistance to advertising, firms often seek to promote their own viral content to boost brand awareness. However, a key shortcoming of virality is its ephemeral nature. To gain insight into sustaining virality, we develop a quasi-experimental approach that estimates the backward spillover onto a focal platform by introducing a piece of content onto a new platform. We posit that introducing content to the audience of a new platform can generate word of mouth, which may affect its consumption within an earlier platform. We estimate these spillovers using data on 381 viral videos on 26 platforms (e.g., YouTube, Vimeo) and observe how consumption of videos on an initial “lead” platform is affected by their subsequent introduction onto “lag” platforms. This spillover is estimated as follows: for each multiplatform video, we compare its view growth after being introduced onto a new platform to that of a synthetic control based on similar single-platform videos. Analysis of 275 such spillover scenarios reveals that introducing a video onto a lag platform roughly doubles its subsequent view growth in the lead platform. This positive cross-platform spillover is persistent, bursty, and strongest in the first 42 days. We find that spillover is boosted when the video is consumed more in the lag platform, when the consumption rate peaks earlier in the lag platform, and when the lag platform targets a foreign market. Our findings suggest that firms can sustain the popularity of their viral content by introducing it onto additional platforms (e.g., Vimeo) after posting it on a focal platform (e.g., YouTube). As a result of their posting on the latter platforms, firms can expect subsequent view growth on the focal platform to roughly double. The aforementioned benefits persists for up to five lag platforms. Platforms should also consider that a positive cross-platform spillover may help platforms reinforce each other’s usage, rather than cannibalize each other.