Causal inference framework for generalizable safety effect estimates

Assessing the Negative Binomial-Lindley model for crash hotspot identification: Insights from Monte Carlo simulation analysis

Identifying hazardous crash sites (or hotspots) is a crucial step in highway safety management. The Negative Binomial (NB) model is the most common model used in safety analyses and evaluations - including hotspot identification. The NB model, however, is not without limitations. In fact, this model does not perform well when data are highly dispersed, include excess zero observations, or have a long tail. Recently, the Negative Binomial-Lindley (NB-L) model has been proposed as an alternative to the NB. The NB-L model overcomes several limitations related to the NB, such as addressing the issue of excess zero observations in highly dispersed data. However, it is not clear how the NB-L model performs regarding the hotspot identification. In this paper, an innovative Monte Carlo simulation protocol was designed to generate a wide range of simulated data characterized by different means, dispersions, and percentage of zeros. Next, the NB-L model was written as a Full-Bayes hierarchical model and compared with the Full-Bayes NB model for hotspot identification using extensive simulation scenarios. Most previous studies focused on statistical fit, and showed that the NB-L model fits the data better than the NB. In this research, however, we investigated the performance of the NB-L model in identifying the hazardous sites. We showed that there is a trade-off between the NB-L and NB when it comes to hotspot identification. Multiple performance metrics were used for the assessment. Among those, the results show that the NB-L model provides a better specificity in identifying hotspots, while the NB model provides a better sensitivity, especially for highly dispersed data. In other words, while the NB model performs better in identifying hazardous sites, the NB-L model performs better, when budget is limited, by not selecting non-hazardous sites as hazardous.

Road safety evaluation with multiple treatments: A comparison of methods based on simulations

This paper focuses on ex-post road safety evaluation with multiple treatments. The potential outcome framework for causal inference is introduced to formalize the causal estimands of interest. Various estimation methods are compared via performing simulation experiments based on semi-synthetic data constructed from a London 20 mph zones dataset. The methods under evaluation include regressions, propensity score (PS) based methods, and a machine learning-based method termed generalized random forests (GRF). Both PS-based methods and GRF show higher flexibility with respect to functional specifications of outcome models. Moreover, GRF shows great superiority in the cases where road safety treatments are assigned following specific criteria and/or where there are heterogeneous treatment effects. Considering the ex-post evaluation of combined effects of multiple treatments has significant practical value, the potential outcome framework and the estimation methods presented in this paper are highly recommended for road safety studies.

Examining the performance of engineering treatment evaluation methodologies using the hypothetical treatment and actual treatment settings

The selection of treatment evaluation methodology is paramount in determining reliable crash modification factors (CMFs) for engineering treatments. A lack of ground truth makes it cumbersome to examine the performance of treatment evaluation methodologies. In addition, a sound methodological framework is critical for evaluating the performances of treatment evaluation methodologies. In addressing these challenges, this study proposed a framework for assessing treatment evaluation methodologies by hypothetical treatments with known ground truth and actual real-world treatments. In particular, this study examined three before-after treatment evaluation approaches: 1) Empirical Bayes, 2) Simulation-based Empirical Bayes, and 3) Full Bayes methods. In addition, this study examined the Cross-Sectional treatment evaluation methodology. The methodological framework utilized five datasets of hypothetical treatment with known ground truth based on the hotspot identification method and a real-world dataset of wide centerline treatment on two-lane, two-way rural highways in Queensland, Australia. Results showed that all the methods could identify the ground truth of hypothetical treatments, but the Full Bayes approach better predicts the known ground truth compared to Empirical Bayes, Simulation-based Empirical Bayes, and Cross-Sectional methods. The Full Bayes approach was also found to provide the most precise estimate for real-world wide centerline treatment along rural highways compared to other methods. Moreover, the current study highlighted that the Cross-Sectional method offers a viable estimate of treatment effectiveness in case the before-period data is limited.

How effective is reducing traffic speed for safer work zones? Methodology and a case study in Pennsylvania

Transportation agencies post and enforce reduced speed limits in work zones to ensure work zone safety, since traffic speed is found to be associated with work zone crash risks. However, prior findings on the relationship between speed and crash rate in work zones are inconsistent. This may be attributed to the methods of statistical associations between traffic speed and crash risks that do not necessarily discover true causal relations. In fact, work zone presence could lead to the reduction of actual traffic speed that influences crash risks, where it may also directly impose effects on crash risks as a result of work zone configurations. The actual traffic speed (not posted speed limit) is also known as a "mediator" where work zones can indirectly impact the crash risks. It is challenging to rigorously separate the causal effect of traffic speed on work zone crash risk from that directly caused by work zones. The underlying causal relation could help to determine what reduced post speed limit (with enforcement) is necessary to ensure work zone safety under the most desired "actual traffic speed". This study proposes to use the sequential g-estimation and the regression discontinuity design to estimate the controlled direct effect of traffic speed on work zone crashes. Two research gaps are identified and filled: inaccurate inferences of the effect of reduced speed limit in work zones as a result of ignoring (1) potential post-treatment bias since traffic speed is a mediator; and (2) potential confounding bias caused by unobservable roadway characteristics. The proposed methodology was applied to 4008 work zones in Pennsylvania from 2015 to 2017, and the results were validated through a series of robustness tests. The results indicate that the direct causal effect of the presence of work zones on crash risk is significantly positive when the traffic speed is relatively low (i.e., lower than 55 mph in this case study), while traffic speed has a positive causal effect on crash occurrences when the actual traffic speed is high (i.e., greater or equal to 55 mph). It suggests that strictly enforcing reduced posted speed limits in work zones is particularly effective when the actual traffic speed is greater than 55 mph. This is particularly true on roadways with high traffic volume (i.e., AADT > 20,000 vehicles per day), long, and daytime work zones (i.e., > 3000 m). On the other hand, the effect of enforcing reduced speed on work zone safety is unclear when the actual speed is already low. In this case, improving work zone configurations and driving behaviors may be more effective in reducing crash risks.

Inferring heterogeneous treatment effects of work zones on crashes

The increasing number of work zone crashes has been a significant concern for road users, transportation agencies, and researchers. Crashes can be caused by work zones, and this effect changes across different work zone configurations, traffic volumes, roadway functional classifications, and weather conditions. This is typically represented by Crash Modification Functions (CMFunctions). However, current methods for developing work zone CMFunctions have two major limitations: (1) They focus on analyzing statistical associations and fail to mitigate the confounding bias due to possible unobservable roadway characteristics; and (2) They cannot address CMFunctions of multiple variables simultaneously, such as weather and traffic conditions, since they are represented using mixed data types (continuous and categorical) that could potentially affect the causal effect of work zones on crashes. In this study, we develop a method that utilizes causal forest with fixed-effect modeling to mitigate the confounding bias while identifying CMFunctions conditioning on various environmental characteristics, including work zone configurations, traffic volume, roadway functional classification, and weather conditions. The developed method was applied to 3378 work zones that occurred in Pennsylvania between 2015 and 2017. The results were validated via a series of robustness tests. The validations demonstrate that this method can mitigate the confounding bias and identify CMFunctions of multiple variables. The results also show that the causal effect of a work zone on crash occurrence is significantly positive (p<0.05) on roadways with high traffic volumes (e.g., > 20,000 vehicles per day) and on medium length (e.g., 2000 to 5000 m) work zones. It appears that having medium-long (e.g., between 6000 and 8000 m) work zones or long duration (e.g., longer than 4 h) work zones do not necessarily lead to extra crashes.

Influence of segmentation approaches on the before-after evaluation of engineering treatments: A hypothetical treatment approach

The segmentation of highways is a fundamental step in estimating crash frequency models and conducting a before-after evaluation of engineering treatments, but the effects of segmentation approaches on the engineering treatment evaluations are not known very well. This study examined the effects of segmentation approaches on the before-after evaluation of engineering treatments. In particular, this study evaluated four segmentation approaches by applying the Empirical Bayes technique to a dataset for which the ground truth was known. Four segmentation approaches included Highway Safety Manual (HSM), Fixed (kilometre post), Fisher's, and K-means segmentation. This study utilized a 440 km stretch of rural two-lane two-way highway in Queensland, Australia, to prepare a dataset with known ground truth. The treatment under evaluation was a hypothetical treatment, which should yield a crash modification factor (CMF) of 1. For assigning hypothetical treatment, a total of fifteen datasets were prepared, including ten datasets based on the random assignment and five datasets based on the hotspot identification method. Following the before-after evaluation using the Empirical Bayes technique, the results showed that HSM and Fixed segmentation approaches predict the ground truth in both dataset types. From random assignment datasets, the estimated CMFs using HSM, Fixed, Fisher's, and K-means segmentation approaches deviated from the true CMF (i.e., 1) by 2.32 %, 5.30 %, 6.08 %, and 8.62 %, respectively. In the case of hotspots, the corresponding deviations of CMFs were 8.57 %, 9.37 %, 28.84 %, and 35.43 %, respectively. Overall, HSM segmentation best identified the actual treatment effect, followed by the Fixed segmentation. If the variables to define homogeneity for HSM segmentation are limited, then Fixed segmentation can yield reliable crash modification factors from the before-after treatment evaluations than the crash-based segmentation approaches.

Propensity score methods for road safety evaluation: Practical suggestions from a simulation study

The propensity score (PS) based method has been increasingly used in road safety evaluation studies. However, several major considerations regarding its implementation arise when using the PS method. First, as is well known, the PS method is 'data hungry' in terms of the number of treated and control units, however, it is sometimes difficult and time-consuming to construct a large sample in road safety studies. It would be helpful to better understand how to choose a proper sample size, as well as the ratio of the number of treated units to the control ones. Second, the criteria used for covariates selection of the PS model were not fully consistent across the existing road safety evaluation studies. Due to the complicated mechanisms behind the implementation of road safety measures and policies, including all relevant covariates that affect both the selection into treatment (i.e., implementation of road safety measures) and the outcomes (i.e., road accidents) is impossible. In this paper, we conduct a simulation study to investigate such issues and provide some practical suggestions for using PS methods in road safety evaluations. The estimator considered in this study is the inverse probability weighting estimator based on the PS. Our results suggest that the bias and variance of the estimated treatment effect will remain stable when the sample size reaches a certain level. A proper sample size is the one that ensures relevant covariates achieve acceptable balance. Regarding the issue of covariates selection, including the covariates that significantly affect the road accidents is recommended, regardless of whether they affect the implementation of road safety measures. This study also proposes practical procedures for using the weighting approach to evaluate the effects of road safety treatments.

Quantification of the impact of traffic incidents on speed reduction: A causal inference based approach

This paper designs a systemic framework to quantify speed reduction induced by traffic incidents using a causal inference framework. The results can provide a reference to traffic managers for evaluating incident severities, thus take proper control measures after the incident in order not to underestimate or overestimate the negative impact. A two-phase scheme is proposed, including impacted region determination and speed reduction quantification. We first propose a Frame Region (FR) method, based on the shockwave propagation, to determine the spatiotemporal impacted region (SIR) using speed map. It is worth-noting that we design a statistical experiment to prove the rationality of congestion threshold selection. Secondly, we introduce a causal inference method for identifying the matched freeway segments. The traffic condition of finally matched freeway segments can be served as non-incident traffic condition of the incident occurred location, which contributes to quantifying the incident impact on speed reduction. We further demonstrate the proposed method in a case study by taking advantage of an incident record and related real freeway speed data in China. An interesting observation is that, along with the freeway segments away from the incident location, the congestion duration time of different freeway segments firstly rises and then decreases. The case study also illustrates the impact of incident on speed lasts almost 3 h and the congestion caused by the incident spreads 11 km, while the average causal effect of incident on all the impacted freeway segments is 42.3 km/h.

Evaluating the speed camera sites selection criteria in the UK

INTRODUCTION

Speed cameras have been implemented to improve road safety over recent decades in the UK. Although the safety impacts of the speed camera have been estimated thoroughly, the criteria for selecting camera sites have rarely been studied. This paper evaluates the current speed camera sites selection criteria in the UK based on safety performance.

METHOD

A total of 332 speed cameras and 2,513 control sites with road traffic accident data are observed from 2002 to 2010. Propensity score matching method and empirical Bayes method are employed and compared to estimate the safety effects of speed cameras under different scenarios.

RESULTS

First, the main characteristics of speed cameras meeting and not meeting the selection criteria are identified. The results indicate that the proximity to school zones and residential neighborhoods, as well as population density, are the main considerations when selecting speed camera sites. Then the official criteria used for selecting camera sites are evaluated, including site length (a stretch of road that has a fixed speed camera or has had one in the past), previous accident history, and risk value (a numerical scale of the risk level). The results suggest that a site length of 500 m should be used to achieve the optimum safety effects of speed cameras. Furthermore, speed cameras are most effective in reducing crashes when the requirement of minimum number of historical killed and seriously injured collisions (KSIs) is met. In terms of the risk value, it is found that the speed cameras can obtain optimal effectiveness with a risk value greater than or equal to 30, rather than the recommended risk value of 22.

A causal analysis of time-varying speed camera safety effects based on the propensity score method

INTRODUCTION

Speed limit enforcement cameras provide an effective approach to reduce vehicle speeds and the number of road accidents. However, it is still unclear whether the safety effects of speed cameras show durability over long periods of time. This paper analyses how the effects of speed cameras on road accidents change over time. A total number of 771 camera sites and 4787 potential control sites are observed for a period of 18 years (1999-2016) across England.

METHOD

Covariates such as road class, crash history, speed limit, and annual average daily traffic (AADT) are included in the data set. A difference in difference (DID) based propensity score matching (PSM) method is employed to select proper control sites and estimate the treatment effects. The safety effects of speed cameras are then evaluated from a long-term perspective. The post-treatment period is divided into four equal-length periods: early, medium 1 and 2, and late.

RESULTS AND CONCLUSIONS

The results show that speed cameras have significantly reduced the number of road accidents near the camera sites. However, the effects vary across different time periods. The safety effects of speed cameras experienced a sharp decrease during the medium periods after an initial period of highly reduced accidents (medium 1: -53.1%, medium 2: -40.7%) and recovered slightly during the late period. In addition, to evaluate the criteria for selecting camera sites in the UK, we further investigated whether speed cameras at high risk sites have better safety performance. The results show that while safety effects at high risk camera sites also decreased during the medium periods, the reduction was smaller (medium 1: -20.8%, medium 2: -2.1%). Practical Applications: Appropriate road traffic regulations and management, as well as proper camera sites selection criterion, are important to maintain the effectiveness of speed cameras.

Empirical Bayes before-after evaluation of horizontal curve warning pavement markings on two-lane rural highways in Pennsylvania

Roadway departure crashes contribute to a large proportion of fatal and injury crashes in the United States. These crash types are more likely to occur along horizontal curve sections of a roadway. Countermeasures that prevent vehicles from departing the roadway is one method to mitigate roadway departure crashes. Pennsylvania has deployed on-pavement horizontal curve warning markings in advance of horizontal curves on two-lane rural highways as a roadway departure crash reduction strategy. This study used an Empirical Bayes (EB) before-after study design to evaluate the safety effects of the horizontal curve warning pavement markings. A total of 263 treatment sites and more than 21,000 reference sites were included in the evaluation. Crash modification factors were developed for total, fatal plus injury, run-off-road, nighttime, nighttime run-off-road, and nighttime fatal plus injury crashes. The point estimates for each of these crashes ranged from 0.65 to 0.77 - the results were statistically significant for total and fatal plus injury crashes at the 95th-percentile confidence level.

Incorporating Bayesian methods into the propensity score matching framework: A no-treatment effect safety analysis

The propensity score matching method has been used to estimate safety countermeasure (treatment) effects from observational crash data. Within the counterfactual framework, propensity score matching is used to balance the covariates between treatment and control groups. Recent studies in traffic safety research have demonstrated the strength of this method in reducing the bias caused by treatment site selection. However, several general issues associated with safety effect estimates may still influence the effectiveness and robustness of this method. In the present study, Bayesian methods were integrated into the propensity score matching method. Bayesian models are known for their ability to capture heterogeneity and modeling uncertainty. This may help mitigate unobserved variable effects in the roadway and crash data. Furthermore, the sampling-based algorithm used for Bayesian estimation yields more consistent estimates in small region analysis than estimates from frequentist modeling. In this study, a dataset that was used to evaluate the safety effects of the dual application of shoulder and centerline rumble strips on two-lane rural highways was acquired. Only data from the before treatment period were used in a no-treatment effect analysis in order to compare the results of a Bayesian propensity score analysis to a frequentist propensity score analysis. Because no treatment was applied during the analysis period, it was assumed that there would be no treatment effect, or a crash modification factor equal to 1.0. The Bayesian propensity score matching method nominally outperformed the frequentist propensity score matching method in the largest sample and produced near-identical results in the medium sample, but neither method closely approximated the assumed, true crash modification factor in the small sample analysis. A simulation study is recommended to further study the effects of sample size and confounding factors when comparing the Bayesian and frequentist propensity score matching methods.

Evaluating the causal effects of cellphone distraction on crash risk using propensity score methods

INTRODUCTION/OBJECTIVE

This paper evaluates the causal effects of cellphone distraction on traffic crashes using propensity score weighting approaches and naturalistic driving study (NDS) data.

METHODS

We adopt three propensity score weighting approaches to estimate the causal odds ratio (OR) of cellphone use on three different event-populations, including average treatment effect (ATE) on the whole population, average treatment effect on the treated population (ATT), and average treatment effect on the overlapping population (ATO). Three types of cellphone distractions are evaluated: overall cellphone use, talking, and visual-manual tasks. The propensity scores are estimated based on driver, roadway, and environmental characteristics. The Second Strategic Highway Research Program NDS data used in this study include 3400 participant drivers with 1047 severe crashes and 19,798 random case-cohort control driving segments.

RESULTS

The study reveals several highly imbalanced potential confounding factors among cellphone use groups, e.g., income, age, and time of day, which could lead to biased risk estimation. All three propensity score approaches improve the balance of the baseline characteristics. The propensity score adjusted ORs differ from unweighted ORs substantially, ranging from -44.25% to 54.88%. Specifically, the adjusted ORs for young drivers are higher than unweighted ORs and these for middle-age drivers are lower. Among different cellphone related distractions, the ORs associated with visual-manual tasks (OR range: 3.47-6.63) are uniformly higher than overall cellphone distraction and cellphone talking (OR range: 0.63-4.15). Cellphone talking increases the risk for young drivers but has no significant impact on middle-age drivers.

CONCLUSION

Propensity score approaches effectively mitigate potential confounding effect caused by imbalanced driver environmental characteristics in the observational NDS data. The ATT and ATO estimands are recommended for NDS case-cohort studies. ATT reflects the effect among exposed events, i.e. crashes or controls with cellphone exposure and ATO reflects the effect among events with similar characteristics. The study confirms the significant causal effect of overall cellphone distraction on crash risk and the heterogeneity in safety impact by age group.

Are multiple speed cameras more effective than a single one? Causal analysis of the safety impacts of multiple speed cameras

Most previous studies investigate the safety effects of a single speed camera, ignoring the potential impacts from adjacent speed cameras. The mutual influence between two or even more adjacent speed cameras is a relevant attribute worth taking into account when evaluating the safety impacts of speed cameras. This paper investigates the safety effects of two or more speed cameras observed within a specific radius which are defined as multiple speed cameras. A total of 464 speed cameras at treated sites and 3119 control sites are observed and related to road traffic accident data from 1999 to 2007. The effects of multiple speed cameras are evaluated using pairwise comparisons between treatment units with different doses based on the propensity score methods. The spatial effect of multiple speed cameras is investigated by testing various radii. There are two major findings in this study. First, sites with multiple speed cameras perform better in reducing the absolute number of road accidents than those with a single camera. Second, speed camera sites are found to be most effective with a radius of 200 m. For a radius of 200 m and 300 m, the reduction in the personal injury collisions by multiple speed cameras are 21.4 % and 13.2 % more than a single camera. Our results also suggest that multiple speed cameras are effective within a small radius (200 m and 300 m).

Comparison of empirical Bayes and propensity score methods for road safety evaluation: A simulation study

Statistical evaluation of road safety interventions can be undertaken using a variety of different approaches, typically requiring different assumptions to obtain causal identification. In this paper, we conduct a simulation study to compare the performance of empirical Bayes (EB) and propensity score (PS) based methods, which have featured prominently in the recent literature, in settings with and without violation of key assumptions. The estimators considered include EB, inverse probability weighting (IPW), and Doubly Robust (DR) estimation. We find that while the EB approach has good finite sample properties when model assumptions are met, the consistency of this estimator is substantially diminished when the reference and treated sites follow different functions. The IPW estimator performs well in large samples, but requires a correctly specified PS model with sufficient overlap in covariate distributions between treated and control units. The DR estimator allows for violation of assumptions in either the regression or PS model, but not both. We find that this added level of robustness affords overall better performance than attained via EB or IPW estimation.

A New Methodology for Before-After Safety Assessment Using Survival Analysis and Longitudinal Data

The widely used empirical Bayes (EB) and full Bayes (FB) methods for before-after safety assessment are sometimes limited because of the extensive data needs from additional reference sites. To address this issue, this study proposes a novel before-after safety evaluation methodology based on survival analysis and longitudinal data as an alternative to the EB/FB method. A Bayesian survival analysis (SARE) model with a random effect term to address the unobserved heterogeneity across sites is developed. The proposed survival analysis method is validated through a simulation study before its application. Subsequently, the SARE model is developed in a case study to evaluate the safety effectiveness of a recent red-light-running photo enforcement program in New Jersey. As demonstrated in the simulation and the case study, the survival analysis can provide valid estimates using only data from treated sites, and thus its results will not be affected by the selection of defective or insufficient reference sites. In addition, the proposed approach can take into account the censored data generated due to the transition from the before period to the after period, which has not been previously explored in the literature. Using individual crashes as units of analysis, survival analysis can incorporate longitudinal covariates such as the traffic volume and weather variation, and thus can explicitly account for the potential temporal heterogeneity.