A simulation-based empirical bayes approach: Incorporating unobserved heterogeneity in the before-after evaluation of engineering treatments

Quantification of safety improvements and human-machine tradeoffs in the transition to automated driving

The assumption of reduced human error-related crashes with increasing levels of automation in pursuing Level 5 automation lacks empirical evidence. As automation levels rise, human error-induced safety hazards are anticipated to decrease, while machine error-induced hazards will increase. However, a quantitative index capturing this tradeoff is absent. Additionally, theoretical modeling of safety improvements during the transition to automated driving remains unexplored, particularly concerning reducing human error-related hazards. These limitations impede the understanding of safety from human and machine perspectives for Automated Vehicle (AV) specialists and manufacturers. This research addresses these gaps by investigating safety performance associations between human and machine factors using the "Human-Machine conflict reduction ratio" (H/M ratio), a novel metric. The study aims to establish safety improvements related to human errors under various automation levels. Sixty participants completed driving tasks on a driving simulator at Levels 0, 4, 3, and 2. Safety performance measures, including conflict frequency and severity, were computed. As a result, Level 4 exhibits the largest decrease (93.3%) compared to manual driving, followed by Level 2 (70.7%) and Level 3 (40.5%). The H/M ratio measures the tradeoff between reducing human and machine error-induced hazards, with Level 2 demonstrating the highest ratio, followed by Levels 4 and 3. Safety performance is evaluated by considering all possible types of human errors at each automation level. Theoretical models from a human factor's perspective are employed to estimate safety improvements at each level. This research contributes to a comprehensive understanding of safety in the "human-machine cooperative driving" phase, offering insights to AV industry practitioners and stakeholders.

Examining the performance of engineering treatment evaluation methodologies using the hypothetical treatment and actual treatment settings

The selection of treatment evaluation methodology is paramount in determining reliable crash modification factors (CMFs) for engineering treatments. A lack of ground truth makes it cumbersome to examine the performance of treatment evaluation methodologies. In addition, a sound methodological framework is critical for evaluating the performances of treatment evaluation methodologies. In addressing these challenges, this study proposed a framework for assessing treatment evaluation methodologies by hypothetical treatments with known ground truth and actual real-world treatments. In particular, this study examined three before-after treatment evaluation approaches: 1) Empirical Bayes, 2) Simulation-based Empirical Bayes, and 3) Full Bayes methods. In addition, this study examined the Cross-Sectional treatment evaluation methodology. The methodological framework utilized five datasets of hypothetical treatment with known ground truth based on the hotspot identification method and a real-world dataset of wide centerline treatment on two-lane, two-way rural highways in Queensland, Australia. Results showed that all the methods could identify the ground truth of hypothetical treatments, but the Full Bayes approach better predicts the known ground truth compared to Empirical Bayes, Simulation-based Empirical Bayes, and Cross-Sectional methods. The Full Bayes approach was also found to provide the most precise estimate for real-world wide centerline treatment along rural highways compared to other methods. Moreover, the current study highlighted that the Cross-Sectional method offers a viable estimate of treatment effectiveness in case the before-period data is limited.

A before-after evaluation of protected right-turn signal phasings by applying Empirical Bayes and Full Bayes approaches with heterogenous count data models

Right-turn crashes (or left-turn crashes for the US or similar countries) represent over 40 % of signalized intersection crashes in Queensland, Australia. Protected right-turn phasings are a widely used countermeasure for right-turn crashes, but the research findings on their effects across different crash types and intersection types are not consistent. Methodologically, the Empirical Bayes and Full Bayes techniques are generally applied for before-after evaluations, but the inclusion of heterogeneous models within these techniques has not been considered much. Addressing these research gaps, the objective of this study is to evaluate the effectiveness of protected right-turn signal phasings at signalized intersections employing heterogeneous count data models with the Empirical Bayes and Full Bayes techniques. In particular, the Empirical Bayes approach based on random parameters Poisson-Gamma models (simulation-based Empirical Bayes), and the Full Bayes approach based on random parameters Poisson-Lognormal intervention models (simulation-based Full Bayes) are applied. A total of 69 Cross intersections (with ten treated sites) and 47 T intersections (with six treated sites) from Southeast Queensland in Australia were included in the analysis to estimate the effects of protected right-turn signal phasings on various crash types. Results show that the change of signal phasing from a permissive right-turn phasing to the protected right-turn phasing at cross and T intersections reduces about 87 % and 91 % of right-turn crashes, respectively. In addition, the effect of protected right-turn phasings on rear-end crashes was not significant. The heterogenous count data models significantly address extra Poisson variation, leading to efficient safety estimates in both simulation-based Empirical Bayes and simulation-based Full Bayes approaches. This study demonstrates the importance of accounting for unobserved heterogeneity for the before-after evaluation of engineering countermeasures.

Influence of segmentation approaches on the before-after evaluation of engineering treatments: A hypothetical treatment approach

The segmentation of highways is a fundamental step in estimating crash frequency models and conducting a before-after evaluation of engineering treatments, but the effects of segmentation approaches on the engineering treatment evaluations are not known very well. This study examined the effects of segmentation approaches on the before-after evaluation of engineering treatments. In particular, this study evaluated four segmentation approaches by applying the Empirical Bayes technique to a dataset for which the ground truth was known. Four segmentation approaches included Highway Safety Manual (HSM), Fixed (kilometre post), Fisher's, and K-means segmentation. This study utilized a 440 km stretch of rural two-lane two-way highway in Queensland, Australia, to prepare a dataset with known ground truth. The treatment under evaluation was a hypothetical treatment, which should yield a crash modification factor (CMF) of 1. For assigning hypothetical treatment, a total of fifteen datasets were prepared, including ten datasets based on the random assignment and five datasets based on the hotspot identification method. Following the before-after evaluation using the Empirical Bayes technique, the results showed that HSM and Fixed segmentation approaches predict the ground truth in both dataset types. From random assignment datasets, the estimated CMFs using HSM, Fixed, Fisher's, and K-means segmentation approaches deviated from the true CMF (i.e., 1) by 2.32 %, 5.30 %, 6.08 %, and 8.62 %, respectively. In the case of hotspots, the corresponding deviations of CMFs were 8.57 %, 9.37 %, 28.84 %, and 35.43 %, respectively. Overall, HSM segmentation best identified the actual treatment effect, followed by the Fixed segmentation. If the variables to define homogeneity for HSM segmentation are limited, then Fixed segmentation can yield reliable crash modification factors from the before-after treatment evaluations than the crash-based segmentation approaches.