Estimating causal effects: considering three alternatives to difference-in-differences estimation

Cigarette Pack Prices and Sales Following Policy Changes in California, 2011-2018

Objectives. To estimate the combined effect of California's Tobacco 21 law (enacted June 2016) and $2-per-pack cigarette excise tax increase (enacted April 2017) on cigarette prices and sales, compared with matched comparator states.Methods. We used synthetic control methods to compare cigarette prices and sales after the policies were enacted, relative to what we would have expected without the policy reforms. To estimate the counterfactual, we matched pre-reform covariate and outcome trends between California and control states to construct a "synthetic" California.Results. Compared with the synthetic control in 2018, cigarette prices in California were $1.89 higher ($7.86 vs $5.97; P < .001), and cigarette sales were 16.6% lower (19.9 vs 16.6 packs per capita; P < .001). This reduction in sales equates to 153.9 million fewer packs being sold between 2017 and 2018.Conclusions. California's new cigarette tax was largely passed on to consumers. The new cigarette tax, combined with the Tobacco 21 law, have contributed to a rapid and substantial reduction in cigarette consumption in California.

Cross-Platform Spillover Effects in Consumption of Viral Content: A Quasi-Experimental Analysis Using Synthetic Controls

To inform product release and distribution strategies, research has analyzed cross-market spillovers in new product adoption. However, models that examine these effects for digital and viral media are still evolving. Given resistance to advertising, firms often seek to promote their own viral content to boost brand awareness. However, a key shortcoming of virality is its ephemeral nature. To gain insight into sustaining virality, we develop a quasi-experimental approach that estimates the backward spillover onto a focal platform by introducing a piece of content onto a new platform. We posit that introducing content to the audience of a new platform can generate word of mouth, which may affect its consumption within an earlier platform. We estimate these spillovers using data on 381 viral videos on 26 platforms (e.g., YouTube, Vimeo) and observe how consumption of videos on an initial “lead” platform is affected by their subsequent introduction onto “lag” platforms. This spillover is estimated as follows: for each multiplatform video, we compare its view growth after being introduced onto a new platform to that of a synthetic control based on similar single-platform videos. Analysis of 275 such spillover scenarios reveals that introducing a video onto a lag platform roughly doubles its subsequent view growth in the lead platform. This positive cross-platform spillover is persistent, bursty, and strongest in the first 42 days. We find that spillover is boosted when the video is consumed more in the lag platform, when the consumption rate peaks earlier in the lag platform, and when the lag platform targets a foreign market. Our findings suggest that firms can sustain the popularity of their viral content by introducing it onto additional platforms (e.g., Vimeo) after posting it on a focal platform (e.g., YouTube). As a result of their posting on the latter platforms, firms can expect subsequent view growth on the focal platform to roughly double. The aforementioned benefits persists for up to five lag platforms. Platforms should also consider that a positive cross-platform spillover may help platforms reinforce each other’s usage, rather than cannibalize each other.

Communication training and the prescribing pattern of antibiotic prescription in primary health care

BACKGROUND

The treatment of upper respiratory tract infections (URTIs) accounts for the majority of antibiotic prescriptions in primary care, although an antibiotic therapy is rarely indicated. Non-clinical factors, such as time pressure and the perceived patient expectations are considered to be reasons for prescribing antibiotics in cases where they are not indicated. The improper use of antibiotics, however, can promote resistance and cause serious side effects. The aim of the study was to clarify whether the antibiotic prescription rate for infections of the upper respiratory tract can be lowered by means of a short (2 x 2.25h) communication training based on the MAAS-Global-D for primary care physicians.

METHODS

In total, 1554 primary care physicians were invited to participate in the study. The control group was formed from observational data. To estimate intervention effects we applied a combination of difference-in-difference (DiD) and statistical matching based on entropy balancing. We estimated a corresponding multi-level logistic regression model for the antibiotic prescribing decision of German primary care physicians for URTIs.

RESULTS

Univariate estimates detected an 11-percentage-point reduction of prescriptions for the intervention group after the training. For the control group, a reduction of 4.7% was detected. The difference between both groups in the difference between the periods was -6.5% and statistically significant. The estimated effects were nearly identical to the effects estimated for the multi-level logistic regression model with applied matching. Furthermore, for the treatment of young women, the impact of the training on the reduction of antibiotic prescription was significantly stronger.

CONCLUSIONS

Our results suggest that communication skills, implemented through a short communication training with the MAAS-Global-D-training, lead to a more prudent prescribing behavior of antibiotics for URTIs. Thereby, the MAAS-Global-D-training could not only avoid unnecessary side effects but could also help reducing the emergence of drug resistant bacteria. As a consequence of our study we suggest that communication training based on the MAAS-Global-D should be applied in the postgraduate training scheme of primary care physicians.

A comparison of methods for health policy evaluation with controlled pre-post designs

Objective

To compare interactive fixed effects (IFE) and generalized synthetic control (GSC) methods to methods prevalent in health policy evaluation and re‐evaluate the impact of the hip fracture best practice tariffs introduced for hospitals in England in 2010.

Data Sources

Simulations and Hospital Episode Statistics.

Study Design

Best practice tariffs aimed to incentivize providers to deliver care in line with guidelines. Under the scheme, 62 providers received an additional payment for each hip fracture admission, while 49 providers did not. We estimate the impact using difference‐in‐differences (DiD), synthetic control (SC), IFE, and GSC methods. We contrast the estimation methods' performance in a Monte Carlo simulation study.

Principal Findings

Unlike DiD, SC, and IFE methods, the GSC method provided reliable estimates across a range of simulation scenarios and was preferred for this case study. The introduction of best practice tariffs led to a 5.9 (confidence interval: 2.0 to 9.9) percentage point increase in the proportion of patients having surgery within 48 hours and a statistically insignificant 0.6 (confidence interval: −1.4 to 0.4) percentage point reduction in 30‐day mortality.

Conclusions

The GSC approach is an attractive method for health policy evaluation. We cannot be confident that best practice tariffs were effective.

Impact of transforming mental health services for young people in England on patient access, resource use and health: a quasi-experimental study

OBJECTIVE

To evaluate the impact of child and adolescent mental health services (CAMHS) transformation in South East England on patient access, resource utilisation and health outcomes.

DESIGN

In an observational study, we use difference-in-differences analysis with propensity score matching to analyse routinely collected patient level data.

SETTING

Three CAMHS services in South East England.

PARTICIPANTS

All patients attending CAMHS between April 2012 and December 2018, with more than 57 000 spells of care included.

MAIN OUTCOME MEASURES

The rate and volume of people accessing CAMHS; waiting times to the first contact and waiting times between the first and second contact; and health outcomes, including the Strengths and Difficulties Questionnaire (SDQ) and the Revised Child Anxiety and Depression Scale (RCADS).

RESULTS

The intervention led to 20% (incidence rate ratio: 1.20; 95% CI: 1.15 to 1.24) more new patients starting per month. There was mixed evidence on waiting times for the first contact. The intervention led to 10% (incidence rate ratio: 1.10; 95% CI: 1.02 to 1.18) higher waiting time for the second contact. The number of contacts per spell (OR: 1.08; 95% CI: 0.94 to 1.25) and the rereferral rate (OR: 1.06; 95% CI: 0.96 to 1.17) were not significantly different. During the post intervention period, patients in the intervention group scored on average 3.3 (95% CI: -5.0 to -1.6) points lower on the RCADS and 1.0 (95% CI: -1.8 to -0.3) points lower on the SDQ compared with the control group after adjusting for the baseline score.

CONCLUSIONS

Overall, there are signs that transformation can help CAMHS achieve the objectives of greater access and improved health outcomes, but trade-offs exist among different performance metrics, particularly between access and waiting times. Commissioners and providers should be conscious of any trade-offs when undertaking service redesign and transformation.

Acute health shocks and labour market outcomes: Evidence from the post crash era

We investigate the labour supply response to an acute health shock for individuals of all working ages, in the post crash era, combining coarsened exact matching and entropy balancing to preprocess data prior to undertaking parametric regression. Identification exploits uncertainty in the timing of an acute health shock, defined by the incidence of cancer, stroke, or heart attack, based on data from Understanding Society. The main finding implies a substantial increase in the baseline probability of labour market exit along with reduced hours and earnings. Younger workers display a stronger labour market attachment than older counterparts, conditional on a health shock. Impacts are stronger for women, older workers, and those who experience more severe limitations and impairments. This is shown to be robust to a broad range of approaches to estimation. Sensitivity tests based on pre-treatment outcomes and using future health shocks as a placebo treatment support our identification strategy.

The 2016 California policy to eliminate nonmedical vaccine exemptions and changes in vaccine coverage: An empirical policy analysis

BACKGROUND

Vaccine hesitancy, the reluctance or refusal to receive vaccination, is a growing public health problem in the United States and globally. State policies that eliminate nonmedical ("personal belief") exemptions to childhood vaccination requirements are controversial, and their effectiveness to improve vaccination coverage remains unclear given limited rigorous policy analysis. In 2016, a California policy (Senate Bill 277) eliminated nonmedical exemptions from school entry requirements. The objective of this study was to estimate the association between California's 2016 policy and changes in vaccine coverage.

METHODS AND FINDINGS

We used a quasi-experimental state-level synthetic control analysis and a county-level difference-in-differences analysis to estimate the impact of the 2016 California policy on vaccination coverage and prevalence of exemptions to vaccine requirements (nonmedical and medical). We used publicly available state-level data from the US Centers for Disease Control and Prevention on coverage of measles, mumps, and rubella (MMR) vaccination, nonmedical exemption, and medical exemption in children entering kindergarten. We used county-level data individually requested from state departments of public health on overall vaccine coverage and exemptions. Based on data availability, we included state-level data for 45 states, including California, from 2011 to 2017 and county-level data for 17 states from 2010 to 2017. The prespecified primary study outcome was MMR vaccination in the state analysis and overall vaccine coverage in the county analysis. In the state-level synthetic control analysis, MMR coverage in California increased by 3.3% relative to its synthetic control in the postpolicy period (top 2 of 43 states evaluated in the placebo tests, top 5%), nonmedical exemptions decreased by 2.4% (top 2 of 43 states evaluated in the placebo tests, top 5%), and medical exemptions increased by 0.4% (top 1 of 44 states evaluated in the placebo tests, top 2%). In the county-level analysis, overall vaccination coverage increased by 4.3% (95% confidence interval [CI] 2.9%-5.8%, p < 0.001), nonmedical exemptions decreased by 3.9% (95% CI 2.4%-5.4%, p < 0.001), and medical exemptions increased by 2.4% (95% CI 2.0%-2.9%, p < 0.001). Changes in vaccination coverage across counties after the policy implementation from 2015 to 2017 ranged from -6% to 26%, with larger increases in coverage in counties with lower prepolicy vaccine coverage. Results were robust to alternative model specifications. The limitations of the study were the exclusion of a subset of US states from the analysis and the use of only 2 years of postpolicy data based on data availability.

CONCLUSIONS

In this study, implementation of the California policy that eliminated nonmedical childhood vaccine exemptions was associated with an estimated increase in vaccination coverage and a reduction in nonmedical exemptions at state and county levels. The observed increase in medical exemptions was offset by the larger reduction in nonmedical exemptions. The largest increases in vaccine coverage were observed in the most "high-risk" counties, meaning those with the lowest prepolicy vaccine coverage. Our findings suggest that government policies removing nonmedical exemptions can be effective at increasing vaccination coverage.

The effect of bariatric surgery on health care costs: A synthetic control approach using Bayesian structural time series

Surgical measures to combat obesity are very effective in terms of weight loss, recovery from diabetes, and improvement in cardiovascular risk factors. However, previous studies found both positive and negative results regarding the effect of bariatric surgery on health care utilization. Using claims data from the largest health insurance provider in Germany, we estimated the causal effect of bariatric surgery on health care costs in a time period ranging from 2 years before to 3 years after bariatric intervention. Owing to the absence of a control group, we employed a Bayesian structural forecasting model to construct a synthetic control. We observed a decrease in medication and physician expenditures after bariatric surgery, whereas hospital expenditures increased in the post-intervention period. Overall, we found a slight increase in total costs after bariatric surgery, but our estimates include a high degree of uncertainty.

Estimating the effects of a studded footwear subsidy program on pedestrian falls among older adults in Gothenburg, Sweden

We study the effects of a studded footwear subsidy program in Gothenburg, Sweden, where a free pair of anti-slip devices was distributed to all residents aged over 65 years as a pedestrian falls prevention measure. Using a difference-in-differences approach with internal age-based controls, we find evidence of a short-term effect on emergency department visits due to slips on snow and ice during the first year of the intervention (-45% [95% CI: -54, -9] in 2013), which equates to 21.8 injuries prevented (95% CI: 3.34, 39.4). A cost-benefit analysis based on this result suggests that the short-term benefits outweigh the total costs of the intervention (benefit-cost ratio: 6.9 [95% CI: 1.05-12.46]), indicating that this type of subsidy program may be an important tool for the prevention of pedestrian falls among older adults during icy weather conditions. However, replication at other sites is recommended before drawing any strong and general conclusions.

The Causal Effect of Access to Finance on Productivity of Small and Medium Enterprises in Vietnam

In many developing countries, obtaining financial services at affordable rates and fair terms has been a significant challenge for small and medium enterprises (SMEs). However, this issue has not been paid much attention in Vietnam, even though SMEs account for about 95% of total enterprises and the financial market of the country has not been well developed. This study investigates the causal effects of access to finance on productivity of SMEs operating in the manufacturing sector in Vietnam. Productivity was measured as the total factor productivity (TFP) obtained by production function estimation using the Levinsohn and Petrin approach. Regarding financial accessibility, two factors covered the extent to which firms might have a bank loan or overdraft facility were employed. To study the causal inferences of access to finance on firm productivity, the research adopted the difference-in-differences (DID) approach, as well as the propensity score matching (PSM) coupled with DID technique. The empirical results indicated that improving the financial accessibility could directly enhance firm productivity. Particularly, it was shown that firms having access to a bank loan could significantly improve TFP by approximately 8.6% in the DID model and about 9% in the PSM-DID model. Meanwhile, the firm average TFP increased by approximately 12.3% and 15.7% in simple DID and PSM-DID models, respectively, when firms had an overdraft facility. These findings suggest that the government should put more effort into assisting SMEs in generating bankable projects, and create a sound and healthy financial environment to stimulate firms’ access to finance, which will ensure their sustainability and growth.

Prescription drug monitoring programs: Assessing the association between "best practices" and opioid use in Medicare

OBJECTIVE

To estimate the impact of implementing prescription drug monitoring program (PDMP) best practices on prescription opioid use.

DATA SOURCES

2007-2012 Medicare claims for noncancer pain patients, and PDMP attributes from the Prescription Drug Abuse Policy System.

STUDY DESIGN

We derived PDMP composite scores using the number of best practices adopted by states (range: 0-14), classifying states as either no PDMP, low strength (0 < score < median), or high strength (score ≥ median). Using generalized linear models, we quantified the association between the PDMP score category and opioid use measures-overall and stratified by disability/age. Sensitivity analyses assessed the general Medicare sample regardless of pain diagnoses, individual PDMP characteristics, and compared GEE model findings to models with state fixed effects.

PRINCIPAL FINDINGS

Compared to non-PDMP states, strong PDMP states had lower opioid cumulative doses (-296 mg; 95% CI: -512, -132), days supplied (-7.84; 95% CI: -10.6, -5.04), prescription fill rates (0.97; 95% CI: 0.95, 0.98), and mean daily doses (-2.31 mg; 95% CI: -3.14, -1.48) but greater prevalence of high opioid doses in disabled adults, whereas there was little or no change in older adults. Findings in states with weak PDMPs were substantively similar to those of strong PDMPs. Results from sensitivity analyses were mostly consistent with main findings except there was a null relationship with mean daily doses and high doses in models with state fixed effects.

CONCLUSIONS

Comprehensive or minimal adoption of PDMP best practices was associated with mostly comparable effects on Medicare beneficiaries' opioid use; however, these effects were concentrated among nonelderly disabled adults.

Long term resource consequences of a nationwide introduction of robotic surgery for women with early stage endometrial cancer

OBJECTIVE

The majority of cost-studies related to robotic surgery has a short follow-up and primarily report the costs from the index surgery. The aim of this study was to evaluate the long-term resource consequences of introducing robotic surgery for early stage endometrial cancer in Denmark.

METHODS

The study included all women with early stage endometrial cancer who underwent robotic, laparoscopic and open access surgery from January 2008 to June 2015. Data was linked from national databases to determine resource consumption and costs from hospital treatments, outpatient contacts, primary health care sector visits, labor market affiliation and prescription of medication. Each patient was observed in a period of 12 months before- and after surgery. The key exposure variable was women who were exposed to robotic surgery compared to those who were not.

RESULTS

A total of 4133 women underwent surgery for early stage endometrial cancer. The study found additional costs of $7309 (95% confidence interval [CI] 2100-11,620, P = 0.001) per patient in the group exposed to robotic surgery including long-term costs post-surgery compared to the non-exposed group (non-robotic group). When controlling for time trends, the introduction of robotic surgery did not reduce the number of bed days (mean diff -0.42, 95% CI -3.03-2.19, P = 0.752).

CONCLUSIONS

The introduction of robotic surgery for early stage endometrial cancer did not generate any long-term cost savings. The additional costs of robotic surgery were primarily driven by the index surgery. Any reduction in bed days could be explained by time trends.

The Effect of Medicaid Expansion on the Nature of New Enrollees’ Emergency Department Use

We examine changes in emergency department (ED) visit acuity and care intensity for uninsured patients who gained Medicaid insurance in 2014 under the Patient Protection and Affordable Care Act. We use 2013-2015 longitudinal patient visit-level data from 30 EDs across 7 states from an emergency medicine group. We examine changes in ED use by previously uninsured Medicaid patients and patients remaining uninsured who were repeat ED users (≥1 visit before and after expansion) using a propensity-score weighted approach with statistical machine learning to estimate the weights. Compared with those remaining uninsured in nonexpansion states, newly covered Medicaid patients in expansion states showed a 29% relative increase in hospital admissions and 32% increase in admissions for nonambulatory care sensitive conditions with no increases in care intensity. Obtaining Medicaid insurance increased the relative proportion of ED visits requiring hospital admission suggesting increased outpatient access for low-acuity conditions previously addressed with ED care.

Exploring the impacts of the 2012 Health and Social Care Act reforms to commissioning on clinical activity in the English NHS: a mixed methods study of cervical screening

OBJECTIVES

Explore the impact of changes to commissioning introduced in England by the Health and Social Care Act 2012 (HSCA) on cervical screening activity in areas identified empirically as particularly affected organisationally by the reforms.

METHODS

Qualitative followed by quantitative methods. Qualitative: semi-structured interviews (with NHS commissioners, managers, clinicians, senior administrative staff from Clinical Commissioning Groups (CCGs), local authorities, service providers), observations of commissioning meetings in two metropolitan areas of England. Quantitative: triple-difference analysis of national administrative data. Variability in the expected effects of HSCA on commissioning was measured by comparing CCGs working with one local authority with CCGs working with multiple local authorities. To control for unmeasured confounders, differential changes over time in cervical screening rates (among women, 25-64 years) between CCGs more and less likely to have been affected by HSCA commissioning organisational change were compared with another outcome-unassisted birth rates-largely unaffected by HSCA changes.

RESULTS

Interviewees identified that cervical screening commissioning and provision was more complex and 'fragmented', with responsibilities less certain, following the HSCA. Interviewees predicted this would reduce cervical screening rates in some areas more than others. Quantitative findings supported these predictions. Areas where CCGs dealt with multiple local authorities experienced a larger decline in cervical screening rates (1.4%) than those dealing with one local authority (1.0%). Over the same period, unassisted deliveries decreased by 1.6% and 2.0%, respectively, in the two groups.

CONCLUSIONS

Arrangements for commissioning and delivering cervical screening were disrupted and made more complex by the HSCA. Areas most affected saw a greater decline in screening rates than others. The fact that this was identified qualitatively and then confirmed quantitatively strengthens this finding. The study suggests large-scale health system reforms may have unintended consequences, and that complex commissioning arrangements may be problematic.

Improving continuity of patient care across sectors: study protocol of a quasi-experimental multi-centre study regarding an admission and discharge model in Germany (VESPEERA)

BACKGROUND

Hospitalisations are a critical event in the care process. Insufficient communication and uncoordinated follow-up care often impede the recovery process of the patient resulting in a high number of rehospitalisations and increased health care costs. The overall aim of this study is the development, implementation and evaluation of a structured programme (VESPEERA) to improve the admission and discharge process.

METHODS

We will conduct an open quasi-experimental multi-centre study with four intervention arms. A cohort selected from insurance claims data will serve as a control group reflecting usual care. The intervention will be implemented in 25 hospital departments and 115 general practices in 9 districts in Baden-Wurttemberg. Eligibility criteria for patients are: age > 18 years, hospital admission or hospitalisation, insurance at the sickness fund "AOK Baden-Wurttemberg", enrolment in general practice-centred care contract. Each study arm will receive different intervention components based on the point of study enrolment and the patient's medical need. The interventions comprise a) a structured assessment in the general practice prior to admission resulting in an admission letter b) a discharge conversation by phone between hospital and general practice, c) a structured assessment and care plan post-discharge and d) telephone monitoring for patients with a high risk of rehospitalisation. The assessments are supported by a software tool ("CareCockpit"), originally developed for structured case management programmes. The primary outcome (rehospitalisation due to the same indication within 90 days) and a range of secondary outcomes (rehospitalisation due to the same indication within 30 days; hospitalisations due to ambulatory care-sensitive conditions; delayed prescription of medication and medical products/ devices and referral to other health practitioner/s after discharge; utilisation of emergency or rescue services within 3 months; average care cost per year and patient participating in the VESPEERA programme) and quality indicators will be determined based on insurance claims data and CareCockpit data. Additionally, a patient survey on satisfaction with cross-sectoral care and health related quality of life will be conducted.

DISCUSSION

Based on the results, area-wide implementation in usual care is well sought. This study will contribute to an improvement of cross-sectoral care during the admission and discharge process.

TRIAL REGISTRATION

DRKS00014294 on DRKS / Universal Trial Number (UTN): U1111-1210-9657, Date of registration 12/06/2018.

Socioeconomic determinants of pediatric asthma emergency department visits under regional economic development in western New York

Although the links between asthma in children and physical environmental factors have been well established, the role of community-level socioeconomic status remains inconclusive. Consequently, little attention has been paid to the dynamic changes in the associations between socioeconomic status and asthma outcomes due to structural changes in the community, such as an influx of financial resources. This study examined the relationship between community-level socioeconomic status indicators and asthma-related emergency department utilization for school-aged children in 2011 and 2015, assessing the early impact of a large-scale regional economic development project in western New York, United States. Our analyses controlled for other community-level health risk factors, such as environmental exposure, and spatial correlation of the emergency department usage data. Results indicated that both median household income and health insurance coverage were key socioeconomic predictors of the children's asthma-related emergency department utilization over the study period. We also found that the risk of emergency department utilization for asthma decreased significantly in the area in which regional economic development projects were completed during the initial stage of the project. Through a comparison study we demonstrated that the spatial correlation present in asthma-related ED utilization improved model fit and corrected biases in the estimates. Although our findings suggest that improving the socioeconomic status of communities contributes to a reduction in emergency department utilization for pediatric asthma, more empirical studies are warranted for evaluating the comprehensive effects of regional economic development on public health.

Now trending: Coping with non-parallel trends in difference-in-differences analysis

Difference-in-differences (DID) analysis is used widely to estimate the causal effects of health policies and interventions. A critical assumption in DID is "parallel trends": that pre-intervention trends in outcomes are the same between treated and comparison groups. To date, little guidance has been available to researchers who wish to use DID when the parallel trends assumption is violated. Using a Monte Carlo simulation experiment, we tested the performance of several estimators (standard DID; DID with propensity score matching; single-group interrupted time-series analysis; and multi-group interrupted time-series analysis) when the parallel trends assumption is violated. Using nationwide data from US hospitals (n = 3737) for seven data periods (four pre-interventions and three post-interventions), we used alternative estimators to evaluate the effect of a placebo intervention on common outcomes in health policy (clinical process quality and 30-day risk-standardized mortality for acute myocardial infarction, heart failure, and pneumonia). Estimator performance was assessed using mean-squared error and estimator coverage. We found that mean-squared error values were considerably lower for the DID estimator with matching than for the standard DID or interrupted time-series analysis models. The DID estimator with matching also had superior performance for estimator coverage. Our findings were robust across all outcomes evaluated.

Synthetic control methodology as a tool for evaluating population-level health interventions

BACKGROUND

Many public health interventions cannot be evaluated using randomised controlled trials so they rely on the assessment of observational data. Techniques for evaluating public health interventions using observational data include interrupted time series analysis, panel data regression-based approaches, regression discontinuity and instrumental variable approaches. The inclusion of a counterfactual improves causal inference for approaches based on time series analysis, but the selection of a suitable counterfactual or control area can be problematic. The synthetic control method builds a counterfactual using a weighted combination of potential control units.

METHODS

We explain the synthetic control method, summarise its use in health research to date, set out its advantages, assumptions and limitations and describe its implementation through a case study of life expectancy following German reunification.

RESULTS

Advantages of the synthetic control method are that it offers an approach suitable when there is a small number of treated units and control units and it does not rely on parallel preimplementation trends like difference in difference methods. The credibility of the result relies on achieving a good preimplementation fit for the outcome of interest between treated unit and synthetic control. If a good preimplementation fit is established over an extended period of time, a discrepancy in the outcome variable following the intervention can be interpreted as an intervention effect. It is critical that the synthetic control is built from a pool of potential controls that are similar to the treated unit. There is currently no consensus on what constitutes a 'good fit' or how to judge similarity. Traditional statistical inference is not appropriate with this approach, although alternatives are available. From our review, we noted that the synthetic control method has been underused in public health.

CONCLUSIONS

Synthetic control methods are a valuable addition to the range of approaches for evaluating public health interventions when randomisation is impractical. They deserve to be more widely applied, ideally in combination with other methods so that the dependence of findings on particular assumptions can be assessed.

Improving care for older people with long-term conditions and social care needs in Salford: the CLASSIC mixed-methods study, including RCT

Background

The Salford Integrated Care Programme (SICP) was a large-scale transformation project to improve care for older people with long-term conditions and social care needs. We report an evaluation of the ability of the SICP to deliver an enhanced experience of care, improved quality of life, reduced costs of care and improved cost-effectiveness.

Objectives

To explore the process of implementation of the SICP and the impact on patient outcomes and costs.

Design

Qualitative methods (interviews and observations) to explore implementation, a cohort multiple randomised controlled trial to assess patient outcomes through quasi-experiments and a formal trial, and an analysis of routine data sets and appropriate comparators using non-randomised methodologies.

Setting

Salford in the north-west of England.

Participants

Older people aged ≥ 65 years, carers, and health and social care professionals.

Interventions

A large-scale integrated care project with three core mechanisms of integration (community assets, multidisciplinary groups and an ‘integrated contact centre’).

Main outcome measures

Patient self-management, care experience and quality of life, and health-care utilisation and costs.

Data sources

Professional and patient interviews, patient self-report measures, and routine quantitative data on service utilisation.

Results

The SICP and subsequent developments have been sustained by strong partnerships between organisations. The SICP achieved ‘functional integration’ through the pooling of health and social care budgets, the development of the Alliance Agreement between four organisations and the development of the shared care record. ‘Service-level’ integration was slow and engagement with general practice was a challenge. We saw only minor changes in patient experience measures over the period of the evaluation (both improvements and reductions), with some increase in the use of community assets and care plans. Compared with other sites, the difference in the rates of admissions showed an increase in emergency admissions. Patient experience of health coaching was largely positive, although the effects of health coaching on activation and depression were not statistically significant. Economic analyses suggested that coaching was likely to be cost-effective, generating improvements in quality of life [mean incremental quality-adjusted life-year gain of 0.019, 95% confidence interval (CI) –0.006 to 0.043] at increased cost (mean incremental total cost increase of £150.58, 95% CI –£470.611 to £711.776).

Limitations

The Comprehensive Longitudinal Assessment of Salford Integrated Care study represents a single site evaluation, with consequent limits on external validity. Patient response rates to the cohort survey were < 40%.

Conclusions

The SICP has been implemented in a way that is consistent with the original vision. However, there has been more rapid success in establishing new integrated structures (such as a formal integrated care organisation), rather than in delivering mechanisms of integration at sufficient scale to have a large impact on patient outcomes.

Future work

Further research could focus on each of the mechanisms of integration. The multidisciplinary groups may require improved targeting of patients or disease subgroups to demonstrate effectiveness. Development of a proven model of health coaching that can be implemented at scale is required, especially one that would provide cost savings for commissioners or providers. Similarly, further exploration is required to assess the longer-term benefits of community assets and whether or not health impacts translate to reductions in care use.

Trial registration

Current Controlled Trials ISRCTN12286422.

Funding

This project was funded by the NIHR Health Services and Delivery Research programme and will be published in full inHealth Services and Delivery Research; Vol. 6, No. 31. See the NIHR Journals Library website for further project information.

Did adolescents in Norway respond to the elimination of copayments for general practitioner services?

Copayments for primary care services may lead to decreased access to and underconsumption of necessary health care for vulnerable patient groups, such as adolescents. In Norway, in 2010, adolescents aged 12 to 15 years were exempted from copayments for general practitioner (GP) services, and the aim of this study is to estimate whether being exempted from copayments led to increases in GP visits. We apply the synthetic control method using the elastic net regression as a data-driven approach to construct a relevant counterfactual from our pool of age groups not affected by the reform. Data on the number of GP consultations for males and females from 2006 to 2013 is obtained from the Norwegian Health Economics Administration. Our findings suggest that exempting adolescents from copayments increased the number of per person GP consultations by 22.1% among females and 13.8% among males. This indicates that adolescents between the ages of 12 and 15 were sensitive to copayments before the reform and females more so than males.

Comparison of Adalimumab and Etanercept for the Treatment of Moderate to Severe Psoriasis: An Indirect Comparison Using Individual Patient Data from Randomized Trials

OBJECTIVES

To compare outcomes between adalimumab and etanercept in the treatment of moderate to severe plaque psoriasis.

METHODS

Study groups included patients randomized to adalimumab or placebo (REVEAL and CHAMPION trials) and those randomized to etanercept or placebo (M10-114 and M10-315 trials). Week 12 outcomes were compared between patients receiving adalimumab and those receiving etanercept after adjusting for cross-trial differences in patient characteristics using propensity score weighting and after subtracting effects of placebo. Outcomes included proportion of patients achieving 75% or more, 90% or more, and 100% reductions from baseline in the Psoriasis Area and Severity Index (PASI75, PASI90, PASI100, respectively), symptom resolution (pruritus = 0; psoriatic pain = 0), lesion resolution (minimal scores for plaque signs erythema, desquamation, and induration, and by body regions head, upper limbs, trunk, and lower limbs), absence of skin-related quality-of-life impact (Dermatology Life Quality Index [DLQI] = 0), "complete disease control" (patient's global assessment [PtGA] = 0), and adverse events.

RESULTS

After adjustment, baseline characteristics were balanced among study groups (adalimumab = 875 vs. placebo = 427; etanercept = 260 vs. placebo = 130). Compared with etanercept, adalimumab was associated with significantly better placebo-adjusted outcomes (PASI75: 62.3% vs. 42.6%; PASI90: 35.9% vs. 12.1%; PASI100: 13.1% vs. 4.9%; pruritus: 24.7% vs. 13.0%; psoriatic pain: 27.4% vs. 8.7%; DLQI: 27.7% vs. 11.7%; and PtGA: 16.4% vs. 10.6%; all P < 0.05), except for similar rates of adverse events and head-specific lesion resolution.

CONCLUSIONS

Compared with etanercept, adalimumab treatment for moderate to severe plaque psoriasis was associated with greater PASI reduction, higher rates of resolution of skin signs and symptoms, and greater improvements in dermatological life quality.

Impact of the North American Free Trade Agreement on high-fructose corn syrup supply in Canada: a natural experiment using synthetic control methods

BACKGROUND:

Critics of free trade agreements have argued that they threaten public health, as they eliminate barriers to trade in potentially harmful products, such as sugar. Here we analyze the North American Free Trade Agreement (NAFTA), testing the hypothesis that lowering tariffs on food and beverage syrups that contain high-fructose corn syrup (HFCS) increased its use in foods consumed in Canada.

METHODS:

We used supply data from the Food and Agriculture Organization of the United Nations to assess changes in supply of caloric sweeteners including HFCS after NAFTA. We estimate the impact of NAFTA on supply of HFCS in Canada using an innovative, quasi-experimental methodology — synthetic control methods — that creates a control group with which to compare Canada’s outcomes. Additional robustness tests were performed for sample, control groups and model specification.

RESULTS:

Tariff reductions in NAFTA coincided with a 41.6 (95% confidence interval 25.1 to 58.2) kilocalorie per capita daily increase in the supply of caloric sweeteners including HFCS. This change was not observed in the control groups, including Australia and the United Kingdom, as well as a composite control of 16 countries. Results were robust to placebo tests and additional sensitivity analyses.

INTERPRETATION:

NAFTA was strongly associated with a marked rise in HFCS supply and likely consumption in Canada. Our study provides evidence that even a seemingly modest change to product tariffs in free trade agreements can substantially alter population-wide dietary behaviour and exposure to risk factors.

Investigation Of Obesity-Related Mortality Rates In Delaware

As Delaware's adult obesity crisis continues to be a leading public health concern, we evaluated Delaware's 1999-2014 vital records to examine the association between obesity and mortality. We used the Delaware population death records from the Centers for Disease Control and Prevention (CDC) WONDER database and the Delaware Health Statistics Center (DHSC). Together with the vital records, we incorporated Microsoft Excel, SAS (Statistical Analysis System) and GIS (geographic information system) tools to analyze obesity influences from county residence, economic status, education, gender, and race. Using the 15-year (1999-2014) time span with the CDC WONDER database, we observed a statistically significant 28.7% increase in the age-adjusted Delaware obesity-related mortality rates (where obesity was a contributory factor). Furthermore, obesity influenced death counts in all three Delaware counties (New Castle, Kent, and Sussex). Kent County experienced the largest increase (66.0%), followed by New Castle County (47.4%), and Sussex County (25.2%). The DHSC mortality rates for all leading causes of death from 2000 to 2011 indicated relatively stable mortality rates for Delaware. However, using CDC WONDER data, the Delaware mortality rate for obesity as a single underlying cause in 2011 was 56.9% higher than mortality rate in 2000.