The effect of gradually lifting the two-child policy on demographic changes in China

Low-fertility rate has been a common problem in many industrialized countries. To reverse the declining trend of new births, Chinese government gradually lifted its restrictions on the number of births per family, allowing for a household to have no more than two children. Little is known about the additional births or population increase contributed by the gradual relaxation of birth restrictions. To fill this gap, this quasi-experimental design study including data from 124 regions used the synthetic control method and controlled interrupted time series analysis to evaluate the differences in birth rates and rates of natural population increase between China and its synthetic control following implementation of the two-child policy from 2011 to 2020. A total of 123 regions were included in the control pool. Data collected during 1990-2010 were used to identify the synthetic China for each study outcome. The mean rate differences of birth rates and rates of natural increase between China and synthetic China after two-child policy implementation were 1.16 per 1000 population and 1.02 per 1000, respectively. These rate differences were distinguished from variation due to chance (one-sided pseudo-P-values: P for birth rates = 0.047, P for rates of natural increase = 0.020). However, there were statistically significant annual reductions in the pre-post trend of birth rates and rates of natural increase compared with those of controls of <0.340 per 1000 population per year [P = 0.007, 95% CI = (-0.584, -0.096)] and <0.274 per 1000 per year [P = 0.028, 95% CI = (-0.518, -0.031)]. The results suggested that lifting birth restrictions had a short-term effect on the increase in birth rates and rates of natural population increase. However, birth policy with lifting birth restrictions alone may not have sustained impact on population growth in the long run.

Estimating treatment effect in randomized trial after control to treatment crossover using external controls

In clinical trials, it is common to design a study that permits the administration of an experimental treatment to participants in the placebo or standard of care group post primary endpoint. This is often seen in the open-label extension phase of a phase III, pivotal study of the new medicine, where the focus is on assessing long-term safety and efficacy. With the availability of external controls, proper estimation and inference of long-term treatment effect during the open-label extension phase in the absence of placebo-controlled patients are now feasible. Within the framework of causal inference, we propose several difference-in-differences (DID) type methods and a synthetic control method (SCM) for the combination of randomized controlled trials and external controls. Our realistic simulation studies demonstrate the desirable performance of the proposed estimators in a variety of practical scenarios. In particular, DID methods outperform SCM and are the recommended methods of choice. An empirical application of the methods is demonstrated through a phase III clinical trial in rare disease.

The augmented synthetic control method in public health and biomedical research

Estimating treatment (or policy or intervention) effects on a single individual or unit has become increasingly important in health and biomedical sciences. One method to estimate these effects is the synthetic control method, which constructs a synthetic control, a weighted average of control units that best matches the treated unit's pre-treatment outcomes and other relevant covariates. The intervention's impact is then estimated by comparing the post-intervention outcomes of the treated unit and its synthetic control, which serves as a proxy for the counterfactual outcome had the treated unit not experienced the intervention. The augmented synthetic control method, a recent adaptation of the synthetic control method, relaxes some of the synthetic control method's assumptions for broader applicability. While synthetic controls have been used in a variety of fields, their use in public health and biomedical research is more recent, and newer methods such as the augmented synthetic control method are underutilized. This paper briefly describes the synthetic control method and its application, explains the augmented synthetic control method and its differences from the synthetic control method, and estimates the effects of an antimalarial initiative in Mozambique using both the synthetic control method and the augmented synthetic control method to highlight the advantages of using the augmented synthetic control method to analyze the impact of interventions implemented in a single region.

Considerations for master protocols using external controls

There has been an increasing use of master protocols in oncology clinical trials because of its efficiency to accelerate cancer drug development and flexibility to accommodate multiple substudies. Depending on the study objective and design, a master protocol trial can be a basket trial, an umbrella trial, a platform trial, or any other form of trials in which multiple investigational products and/or subpopulations are studied under a single protocol. Master protocols can use external data and evidence (e.g. external controls) for treatment effect estimation, which can further improve efficiency of master protocol trials. This paper provides an overview of different types of external controls and their unique features when used in master protocols. Some key considerations in master protocols with external controls are discussed including construction of estimands, assessment of fit-for-use real-world data, and considerations for different types of master protocols. Similarities and differences between regular randomized controlled trials and master protocols when using external controls are discussed. A targeted learning-based causal roadmap is presented which constitutes three key steps: (1) define a target statistical estimand that aligns with the causal estimand for the study objective, (2) use an efficient estimator to estimate the target statistical estimand and its uncertainty, and (3) evaluate the impact of causal assumptions on the study conclusion by performing sensitivity analyses. Two illustrative examples for master protocols using external controls are discussed for their merits and possible improvement in causal effect estimation.

The Long-Term Impacts of an Integrated Care Programme on Hospital Utilisation among Older Adults in the South of England: A Synthetic Control Study

Introduction

Reducing hospital use is often viewed as a possible positive consequence of introducing integrated care (IC). We investigated the impact of an IC programme in North East Hampshire and Farnham (NEHF), in southern England, on hospital utilisation among older adults over a 55 months period.

Method

We used a Generalised Synthetic Control design to investigate the effect of implementing IC in NEHF between 2015 and 2020. For a range of hospital use outcomes, we estimated the trajectory that each would have followed in the absence of IC and compared it with the actual trajectory to estimate the potential impact of IC.

Results

Three years into the programme, emergency admission rates started reducing in NEHF relative to its synthetic control, particularly those resulting in overnight hospital stays. By year 5 of the study overall emergency admission rates were 9.8% lower (95% confidence interval: -17.2% to -0.6%). We found no sustained difference in rates of emergency department (ED) visits, and average length of hospital stay was significantly higher from year 2.

Conclusion

An IC programme in NEHF led to lower than estimated emergency admission rates; however, the interpretation of the impact of IC on admissions is complicated as lower rates did not appear until three years into the programme and the reliability of the synthetic control weakens over a long time horizon. There was no sustained change in ED visit rates.

Confounding adjustment in the analysis of augmented randomized controlled trial with hybrid control arm

The augmented randomized controlled trial (RCT) with hybrid control arm includes a randomized treatment group (RT), a smaller randomized control group (RC), and a large synthetic control (SC) group from real-world data. This kind of trial is useful when there is logistics and ethics hurdle to conduct a fully powered RCT with equal allocation, or when it is necessary to increase the power of the RCT by incorporating real-world data. A difficulty in the analysis of augmented RCT is that the SC and RC may be systematically different in the distribution of observed and unmeasured confounding factors, causing bias when the two control groups are analyzed together as hybrid controls. We propose to use propensity score (PS) analysis to balance the observed confounders between SC and RC. The possible bias caused by unmeasured confounders can be estimated and tested by analyzing propensity score adjusted outcomes from SC and RC. We also propose a partial bias correction (PBC) procedure to reduce bias from unmeasured confounding. Extensive simulation studies show that the proposed PS + PBC procedures can improve the efficiency and statistical power by effectively incorporating the SC into the RCT data analysis, while still control the estimation bias and Type I error inflation that might arise from unmeasured confounding. We illustrate the proposed statistical procedures with data from an augmented RCT in oncology.

Contraceptive access reform and abortion: Evidence from Delaware

OBJECTIVE

To examine the effects of a comprehensive contraceptive access reform, Delaware Contraceptive Access Now, on abortion-one of the most common outcomes of unintended pregnancy.

DATA SOURCE

We used abortion data by state of residence from the Abortion Surveillance System, published by the Centers for Disease Control and Prevention. Our data covers 5 years prior to (2010-2014) and 5 years after the intervention (2015-2019).

STUDY DESIGN

We used synthetic control methods to estimate program effects. Our design compares Delaware to a weighted average of 45 control states ("synthetic Delaware"), where the quality of the comparison is assessed by its similarity to Delaware in pre-period outcome levels and trends.

DATA COLLECTION/EXTRACTION METHODS

Not applicable. We relied on secondary sources.

PRINCIPAL FINDINGS

We did not find statistically significant evidence that the program reduced abortion rates (0.61 fewer abortions per 1000 women, p-value = 0.74) on average, during the intervention period. The treatment effects were slightly larger in 2016 and 2017 (1.97 fewer abortions per 1000 women but not statistically significant) and attenuated in 2018 and 2019. This does not rule out program benefits in easing barriers to contraceptive methods or in reducing unplanned births. However, findings do suggest that increasing contraceptive access might not be an adequate substitute for restricted abortion access resulting from Dobbs v. Jackson Women's Health Organization.

CONCLUSIONS

Our results suggest that comprehensive efforts to improve contraceptive access may not reduce the need for accessible and affordable abortion care.

A comparison of synthetic control approaches for the evaluation of policy interventions using observational data: Evaluating the impact of redesigning urgent and emergency care in Northumberland

OBJECTIVE

To compare the original synthetic control (OSC) method with alternative approaches (Generalized [GSC], Micro [MSC], and Bayesian [BSC] synthetic control methods) and re-evaluate the impact of a significant restructuring of urgent and emergency care in Northeast England, which included the opening of the UK's first purpose-built specialist emergency care hospital.

DATA SOURCES

Simulations and data from Secondary Uses Service data, a single comprehensive repository for patient-level health care data in England.

STUDY DESIGN

Hospital use of individuals exposed and unexposed to the restructuring is compared. We estimate the impact using OSC, MSC, BSC, and GSC applied at the general practice level. We contrast the estimation methods' performance in a Monte Carlo simulation study.

DATA COLLECTION/EXTRACTION METHODS

Hospital activity data from Secondary Uses Service for patients aged over 18 years registered at a general practice in England from April 2011 to March 2019.

PRINCIPAL FINDINGS

None of the methods dominated all simulation scenarios. GSC was generally preferred. In contrast to an earlier evaluation that used OSC, GSC reported a smaller impact of the opening of the hospital on Accident and Emergency (A&E) department (also known as emergency department or casualty) visits and no evidence for any impact on the proportion of A&E patients seen within 4 h.

CONCLUSIONS

The simulation study highlights cases where the considered methods may lead to biased estimates in health policy evaluations. GSC was found to be the most reliable method of those considered. Considering more disaggregated data over a longer time span and applying GSC indicates that the specialist emergency care hospitals in Northumbria had less impact on A&E visits and waiting times than suggested by the original evaluation which applied OSC to more aggregated data.

Use of a real-world synthetic control arm for direct comparison of lisocabtagene maraleucel and conventional therapy in relapsed/refractory large B-cell lymphoma

This study used a real-world population as a synthetic comparator for the single-arm TRANSCEND NHL 001 study (TRANSCEND; NCT02631044) to evaluate the efficacy of lisocabtagene maraleucel (liso-cel) compared with conventional (noncellular) therapies in patients with relapsed/refractory (R/R) large B-cell lymphoma (LBCL). Inclusion and exclusion criteria for the real-world study closely matched the enrollment criteria in TRANSCEND. The analytic comparator cohort was created by matching and balancing observed baseline characteristics of real-world patients with those in TRANSCEND using propensity score methodology. Efficacy outcomes comparing liso-cel- (n = 257) and conventional therapy-treated (n = 257) patients, respectively, significantly favored liso-cel: overall response rate (74% vs 39%; p < 0.0001), complete response rate (50% vs 24%; p < 0.0001), median overall survival (23.5 vs 6.8 months; p < 0.0001), and median progression-free survival (3.5 vs 2.2 months; p < 0.0001). These results demonstrated a statistically significant and clinically meaningful benefit of liso-cel in patients with third- or later-line R/R LBCL relative to conventional therapies.Clinical trial registration: ClinicalTrials.gov identifier: NCT02631044.

Health effects of a ban on late-night alcohol sales

This paper studies the impact of a ban on late-night off-premise alcohol sales between 10 p.m. and 5 a.m. in Germany. We use three large administrative data sets: (i) German diagnosis related groups-Statistik, (ii) data from a large social health insurance, and (iii) Road Traffic Accident Statistics. Applying difference-in-differences and synthetic-control-group methods, we find that the ban had no effects on alcohol-related road casualties, but significantly reduced alcohol-related hospitalizations (doctor visits) among young people by around 9 (18) percent. The decrease is driven by fewer hospitalizations due to acute alcohol intoxication during the night-when the ban is in place-but not during the day.

Cloudy with a chance of munchies: Assessing the impact of recreational marijuana legalization on obesity

Obesity in the US arguably constitutes the most significant health epidemic over the past century. Recent legislative changes allowing for recreational marijuana use further create a need to better understand the relationship between marijuana use and health choices, leading to obesity. We examine this relationship by using a synthetic control approach to examine the impact of legalized recreational marijuana access on obesity rates by comparing Washington State to a synthetically constructed counterfactual. We find that recreational marijuana's introduction did not lead to increased obesity rates and may have led to decreases in obesity.

The effect of increasing Women's autonomy on primary and repeated caesarean sections in Brazil

Caesarean section (C-section) rates continue to rise globally. Yet, there is little consensus about the key determinants of rising C-section rates and the sources of variation in C-section rates across the world. While C-sections can save lives when medically justified, unnecessary surgical procedures can be harmful for women and babies. We show that a state-wide law passed in São Paulo (Brazil), which increased women's autonomy to choose to deliver via C-section even when not medically necessary, is associated with a 3% increase in overall C-section rates. This association was driven by a 5% increase in primary C-sections, rather than repeated C-sections. Since the law emphasizes women's autonomy, these results are consistent with mothers' demand being an important contributor to high C-section rates in this context.

External control cohorts for the single-arm LIBRETTO-001 trial of selpercatinib in RET+ non-small-cell lung cancer

Background

Data for selpercatinib [a selective REarranged during Transfection (RET) inhibitor] from a single-arm trial (LIBRETTO-001, NCT03157128) in RET-fusion-positive advanced/metastatic non-small-cell lung cancer (NSCLC) were used in combination with external data sources to estimate comparative efficacy [objective response rate (ORR), progression-free survival, and overall survival (OS)] in first- and second-line treatment settings.

Methods

Patient-level data were obtained from a de-identified real-world database. Patients diagnosed with advanced/metastatic NSCLC with no prior exposure to a RET inhibitor and one or more prior line of therapy were eligible. Additionally, individual patient-level data (IPD) were obtained from the pemetrexed + platinum arm of KEYNOTE-189 (NCT03950674, first line) and the docetaxel arm of REVEL (NCT01168973, post-progression). Patients were matched using entropy balancing, doubly robust method, and propensity score approaches. For patients with unknown/negative RET status, adjustment was made using a model fitted to IPD from a real-world database.

Results

In first-line unadjusted analyses of the real-world control, ORR was 87.2% for LIBRETTO-001 versus 66.7% for those with RET-positive NSCLC (P = 0.06). After adjustment for unknown RET status and other patient characteristics, selpercatinib remained significantly superior versus the real-world control for all outcomes (all P < 0.001 except unadjusted RET-fusion-positive cohort). Similarly, outcomes were significantly improved versus clinical trial controls (all P < 0.05).

Conclusions

Findings suggest improvement in outcomes associated with selpercatinib treatment versus the multiple external control cohorts, but should be interpreted with caution. Data were limited by the rarity of RET, lack of mature OS data, and uncertainty from assumptions to create control arms from external data.

•

Single-arm trials are limited by the lack of a comparison arm, and external controls are needed.

•

Multiple methodological approaches with various external control arms evaluated the comparative efficacy of selpercatinib.

•

Findings suggest that selpercatinib is associated with significantly improved clinical outcomes versus standard therapies.

•

Results should be considered exploratory and hypothesis generating due to the limitations of this study.

Have Paved Trails and Protected Bike Lanes Led to More Bicycling in Atlanta?: A Generalized Synthetic-Control Analysis

BACKGROUND

Bicycling is an important form of physical activity in populations. Research assessing the effect of infrastructure on bicycling with high-resolution smartphone data is emerging in several places, but it remains limited in low-bicycling US settings, including the Southeastern US. The Atlanta area has been expanding its bicycle infrastructure, including off-street paved trails such as the Atlanta BeltLine and some protected bike lanes.

METHODS

Using the generalized synthetic-control method, we estimated effects of five groups of off-street paved trails and protected bike lanes on bicycle ridership in their corresponding areas. To measure bicycling, we used 2 years (October 1, 2016 to September 30, 2018) of monthly Strava data in Atlanta's urban core along with data from 15 on-the-ground counters to adjust for spatiotemporal variation in app use.

RESULTS

Considering all infrastructure as one joint intervention, an estimated 1.10 (95% confidence interval [CI]: 0.99, 1.18) times more bicycle-distance was ridden than would have been expected in the same areas had the infrastructure not been built, when defining treatment areas by the narrower of two definitions (defined in text). The Atlanta BeltLine Westside Trail and Proctor Creek Greenway had especially strong effect estimates, e.g., ratios of 1.45 (95% CI: 1.12, 1.86) and 1.55 (1.10, 2.14) under each treatment-area definition, respectively. We estimated that other infrastructure had weaker positive or no effects on bicycle-distance ridden.

CONCLUSIONS

This study advances research on the topic because of its setting in the US Southeast, simultaneous assessment of several infrastructure groups, and data-driven approach to estimating effects. See video abstract at, http://links.lww.com/EDE/B936.

Public health implications of opening National Football League stadiums during the COVID-19 pandemic

Using data from 2020, we measure the public health impact of allowing fans into sports stadiums during the COVID-19 pandemic; these results may inform future policy decisions regarding large outdoor gatherings during public health crises. Second, we demonstrate the utility of robust synthetic control in this context. Synthetic control and other statistical approaches may be used to exploit the underlying low-dimensional structure of the COVID-19 data and serve as useful instruments in analyzing the impact of mitigation strategies adopted by different communities. As with all statistical methods, reliable outcomes depend on proper implementation strategies and well-established robustness tests; in the absence of these safeguards, these statistical methods are likely to produce specious or misleading conclusions.

Using attendance data from the 2020 National Football League (NFL) regular season and local COVID-19 case counts, we estimate the public health impact of opening NFL stadiums to fans during the COVID-19 pandemic. Data are analyzed using robust synthetic control, a statistical method that is employed to obtain counterfactual estimates from observational data. Unlike previous studies [J. Kurland et al., SSRN, 2021], which do not consider confounding factors such as evolving policy landscapes in different states, the synthetic control methodology allows us to account for effects that are county specific and may be changing over time. We find it is likely that opening stadiums had no impact on local COVID-19 case counts; this suggests that, for the 2020 NFL season, the benefits of providing a tightly controlled outdoor spectating environment—including masking and distancing requirements—counterbalanced the risks associated with opening. These results are specific to the 2020 NFL season, and care should be taken in generalizing our conclusions. In particular, 1) these data reflect a period during which earlier strains of COVID-19 were dominant prior to the emergence of more-transmissive strains such as the Delta and Omicron variants, and 2) the data are restricted to outdoor environments; hence our results cannot be applied to small indoor spaces where transmission-restricting controls are essential.

Combining Matching and Synthetic Control to Trade off Biases from Extrapolation and Interpolation

The synthetic control (SC) method is widely used in comparative case studies to adjust for differences in pre-treatment characteristics. SC limits extrapolation bias at the potential expense of interpolation bias, whereas traditional matching estimators have the opposite properties. This complementarity motives us to propose a matching and synthetic control (or MASC) estimator as a model averaging estimator that combines the standard SC and matching estimators. We show how to use a rolling-origin cross-validation procedure to train the MASC to resolve trade-offs between interpolation and extrapolation bias. We use a series of empirically-based placebo and Monte Carlo simulations to shed light on when the SC, matching, MASC and penalized SC estimators do (and do not) perform well. Then, we apply these estimators to examine the economic costs of conflicts in the context of Spain.

California's Mental Health Services Act and Mortality Due to Suicide, Homicide, and Acute Effects of Alcohol: A Synthetic Control Application

California's Mental Health Services Act (MHSA) substantially expanded funding of county mental health services through a state tax, and led to broad prevention efforts and intensive services for individuals experiencing serious mental disorders. We estimated the associations between MHSA and mortality due to suicide, homicide, and acute effects of alcohol. Using annual cause-specific mortality data for each US state and the District of Columbia from 1976-2015, we used a generalization of the quasi-experimental synthetic control method to predict California's mortality rate for each outcome in the absence of MHSA using a weighted combination of comparison states. We calculated the association between MHSA and each outcome as the absolute difference and percentage difference between California's observed and predicted average annual rates over the postintervention years (2007-2015). MHSA was associated with modest decreases in average annual rates of homicide (-0.81/100,000 persons, corresponding to a 13% reduction) and mortality from acute alcohol effects (-0.35/100,000 persons, corresponding to a 12% reduction). Placebo test inference suggested that the associations were unlikely to be due to chance. MHSA was not associated with suicide. Protective associations with mortality due to homicide and acute alcohol effects provide evidence for modest health benefits of MHSA at the population level.

Can we mitigate the effect of natural disasters on child health? Evidence from the Indian Ocean tsunami in Indonesia

The 2004 Indian Ocean tsunami was an international natural disaster unlike any seen before, killing 166,561 people in Aceh province, Indonesia. It prompted an unprecedented humanitarian response and was a catalyst in ending almost 30 years of civil conflict in Aceh. Since the tsunami was followed by a multitude of events, we first conduct a systematic review to identify those events in Indonesia. We then use a synthetic control method to estimate the combination of those effects on child mortality indicators in Aceh for the 13 years that followed the disaster using data from 258,918 children born between 1990 and 2017. The results show a significant increase in under-5 mortality only the year after the tsunami and no effect in the medium term. However, younger and older children were affected differently in the medium term. In fact, we show a decrease in child mortality among children aged 1-4 years. In contrast, we observe an increase in mortality among children under-1 in 2009 and 2010. Overall, the resilience of Aceh province points to the importance of coordinated international disaster responses after natural disasters.

Free prescriptions for low-income pensioners? The cost of returning to free-of-charge drugs in the Spanish National Health Service

This study estimated the impact of reducing a capped low coinsurance rate for outpatient medicines to nil for low-income pensioners and disabled individuals in the Valencian Community (Spain). This reduction was implemented in January 2016 as a regional reform which modified the national cost-sharing reform adopted in July 2012. The impact of this intervention on the number of monthly prescriptions dispensed between July 2012 and December 2018 was estimated using two different approaches of the synthetic control method, the classical method and the method based on Bayesian structural time series. The estimates from both methods were similar, showing significant overall increases of 6.34% and 6.70% [95% credible interval: 4.05, 9.47], respectively in the number of prescriptions dispensed in this region. These results are similar to those of the previous studies indicating that reducing price from a small amount to zero discontinuously boosts demand. This evidence indicates that the impact of this intervention on the budget of the regional health service is far greater than the amount of the subsidy in the public budget. These results are useful for making accurate budgetary projections for similar eliminations of charges for low-income pensioners in the Spanish National Health Service.

Statistical Properties of Stepped Wedge Cluster-Randomized Trials in Infectious Disease Outbreaks

Randomized controlled trials are crucial for the evaluation of interventions such as vaccinations, but the design and analysis of these studies during infectious disease outbreaks is complicated by statistical, ethical, and logistical factors. Attempts to resolve these complexities have led to the proposal of a variety of trial designs, including individual randomization and several types of cluster randomization designs: parallel-arm, ring vaccination, and stepped wedge designs. Because of the strong time trends present in infectious disease incidence, however, methods generally used to analyze stepped wedge trials might not perform well in these settings. Using simulated outbreaks, we evaluated various designs and analysis methods, including recently proposed methods for analyzing stepped wedge trials, to determine the statistical properties of these methods. While new methods for analyzing stepped wedge trials can provide some improvement over previous methods, we find that they still lag behind parallel-arm cluster-randomized trials and individually randomized trials in achieving adequate power to detect intervention effects. We also find that these methods are highly sensitive to the weighting of effect estimates across time periods. Despite the value of new methods, stepped wedge trials still have statistical disadvantages compared with other trial designs in epidemic settings.

Impact of Regression to the Mean on the Synthetic Control Method: Bias and Sensitivity Analysis

To make informed policy recommendations from observational panel data, researchers must consider the effects of confounding and temporal variability in outcome variables. Difference-in-difference methods allow for estimation of treatment effects under the parallel trends assumption. To justify this assumption, methods for matching based on covariates, outcome levels, and outcome trends-such as the synthetic control approach-have been proposed. While these tools can reduce bias and variability in some settings, we show that certain applications can introduce regression to the mean (RTM) bias into estimates of the treatment effect. Through simulations, we show RTM bias can lead to inflated type I error rates and bias toward the null in typical policy evaluation settings. We develop a novel correction for RTM bias that allows for valid inference and show how this correction can be used in a sensitivity analysis. We apply our proposed sensitivity analysis to reanalyze data concerning the effects of California's Proposition 99, a large-scale tobacco control program, on statewide smoking rates.

Overstated carbon emission reductions from voluntary REDD+ projects in the Brazilian Amazon

Reducing emissions from deforestation and forest degradation (REDD+) has gained international attention over the past decade, as manifested in both United Nations policy discussions and hundreds of voluntary projects launched to earn carbon-offset credits. There are ongoing discussions about whether and how projects should be integrated into national climate change mitigation efforts under the Paris Agreement. One consideration is whether these projects have generated additional impacts over and above national policies and other measures. To help inform these discussions, we compare the crediting baselines established ex-ante by voluntary REDD+ projects in the Brazilian Amazon to counterfactuals constructed ex-post based on the quasi-experimental synthetic control method. We find that the crediting baselines assume consistently higher deforestation than counterfactual forest loss in synthetic control sites. This gap is partially due to decreased deforestation in the Brazilian Amazon during the early implementation phase of the REDD+ projects considered here. This suggests that forest carbon finance must strike a balance between controlling conservation investment risk and ensuring the environmental integrity of carbon emission offsets. Relatedly, our results point to the need to better align project- and national-level carbon accounting.

Synthetic and External Controls in Clinical Trials - A Primer for Researchers

There has been a rapid expansion in the use of non-randomized evidence in the regulatory approval of treatments globally. An emerging set of methodologies have been utilized to provide greater insight into external control data used for these purposes, collectively known as synthetic control methods. Through this paper, we provide the reader with a set of key questions to help assess the quality of literature publications utilizing synthetic control methodologies. Common challenges and real-life examples of synthetic controls are provided throughout, alongside a critical appraisal framework with which to assess future publications.

The European Union Emissions Trading System reduced CO2 emissions despite low prices

International carbon markets are an appealing and increasingly popular tool to regulate carbon emissions. By putting a price on carbon, carbon markets reshape incentives faced by firms and reduce the value of emissions. How effective are carbon markets? Observers have tended to infer their effectiveness from market prices. The general belief is that a carbon market needs a high price in order to reduce emissions. As a result, many observers remain skeptical of initiatives such as the European Union Emissions Trading System (EU ETS), whose price remained low (compared to the social cost of carbon). In this paper, we assess whether the EU ETS reduced [Formula: see text] emissions despite low prices. We motivate our study by documenting that a carbon market can be effective if it is a credible institution that can plausibly become more stringent in the future. In such a case, firms might cut emissions even though market prices are low. In fact, low prices can be a signal that the demand for carbon permits weakens. Thus, low prices are compatible with successful carbon markets. To assess whether the EU ETS reduced carbon emissions even as permits were cheap, we estimate counterfactual carbon emissions using an original sectoral emissions dataset. We find that the EU ETS saved about 1.2 billion tons of [Formula: see text] between 2008 and 2016 (3.8%) relative to a world without carbon markets, or almost half of what EU governments promised to reduce under their Kyoto Protocol commitments. Emission reductions in sectors covered under the EU ETS were higher.

Novel methods for the analysis of stepped wedge cluster randomized trials

Stepped wedge cluster randomized trials (SW-CRTs) have become increasingly popular and are used for a variety of interventions and outcomes, often chosen for their feasibility advantages. SW-CRTs must account for time trends in the outcome because of the staggered rollout of the intervention. Robust inference procedures and nonparametric analysis methods have recently been proposed to handle such trends without requiring strong parametric modeling assumptions, but these are less powerful than model-based approaches. We propose several novel analysis methods that reduce reliance on modeling assumptions while preserving some of the increased power provided by the use of mixed effects models. In one method, we use the synthetic control approach to find the best matching clusters for a given intervention cluster. Another method makes use of within-cluster crossover information to construct an overall estimator. We also consider methods that combine these approaches to further improve power. We test these methods on simulated SW-CRTs, describing scenarios in which these methods have increased power compared with existing nonparametric methods while preserving nominal validity when mixed effects models are misspecified. We also demonstrate theoretical properties of these estimators with less restrictive assumptions than mixed effects models. Finally, we propose avenues for future research on the use of these methods; motivation for such research arises from their flexibility, which allows the identification of specific causal contrasts of interest, their robustness, and the potential for incorporating covariates to further increase power. Investigators conducting SW-CRTs might well consider such methods when common modeling assumptions may not hold.

A comparison of methods for health policy evaluation with controlled pre-post designs

Objective

To compare interactive fixed effects (IFE) and generalized synthetic control (GSC) methods to methods prevalent in health policy evaluation and re‐evaluate the impact of the hip fracture best practice tariffs introduced for hospitals in England in 2010.

Data Sources

Simulations and Hospital Episode Statistics.

Study Design

Best practice tariffs aimed to incentivize providers to deliver care in line with guidelines. Under the scheme, 62 providers received an additional payment for each hip fracture admission, while 49 providers did not. We estimate the impact using difference‐in‐differences (DiD), synthetic control (SC), IFE, and GSC methods. We contrast the estimation methods' performance in a Monte Carlo simulation study.

Principal Findings

Unlike DiD, SC, and IFE methods, the GSC method provided reliable estimates across a range of simulation scenarios and was preferred for this case study. The introduction of best practice tariffs led to a 5.9 (confidence interval: 2.0 to 9.9) percentage point increase in the proportion of patients having surgery within 48 hours and a statistically insignificant 0.6 (confidence interval: −1.4 to 0.4) percentage point reduction in 30‐day mortality.

Conclusions

The GSC approach is an attractive method for health policy evaluation. We cannot be confident that best practice tariffs were effective.