Real-Time Evaluation of Perception Uncertainty and Validity Verification of Autonomous Driving

Deep neural network algorithms have achieved impressive performance in object detection. Real-time evaluation of perception uncertainty from deep neural network algorithms is indispensable for safe driving in autonomous vehicles. More research is required to determine how to assess the effectiveness and uncertainty of perception findings in real-time.This paper proposes a novel real-time evaluation method combining multi-source perception fusion and deep ensemble. The effectiveness of single-frame perception results is evaluated in real-time. Then, the spatial uncertainty of the detected objects and influencing factors are analyzed. Finally, the accuracy of spatial uncertainty is validated with the ground truth in the KITTI dataset. The research results show that the evaluation of perception effectiveness can reach 92% accuracy, and a positive correlation with the ground truth is found for both the uncertainty and the error. The spatial uncertainty is related to the distance and occlusion degree of detected objects.

Geocoding Error, Spatial Uncertainty, and Implications for Exposure Assessment and Environmental Epidemiology

Although environmental epidemiology studies often rely on geocoding procedures in the process of assigning spatial exposure estimates, geocoding methods are not commonly reported, nor are consequent errors in exposure assignment explored. Geocoding methods differ in accuracy, however, and, given the increasing refinement of available exposure models for air pollution and other exposures, geocoding error may account for an increasingly larger proportion of exposure misclassification. We used residential addresses from a reasonably large, dense dataset of asthma emergency department visits from all New York City hospitals (n = 21,183; 26.9 addresses/km²), and geocoded each using three methods (Address Point, Street Segment, Parcel Centroid). We compared missingness and spatial patterning therein, quantified distance and directional errors, and quantified impacts on pollution exposure estimates and assignment to Census areas for sociodemographic characterization. Parcel Centroids had the highest overall missingness rate (38.1%, Address Point = 9.6%, Street Segment = 6.1%), and spatial clustering in missingness was significant for all methods, though its spatial patterns differed. Street Segment geocodes had the largest mean distance error (µ = 29.2 (SD = 26.2) m; vs. µ = 15.9 (SD = 17.7) m for Parcel Centroids), and the strongest spatial patterns therein. We found substantial over- and under-estimation of pollution exposures, with greater error for higher pollutant concentrations, but minimal impact on Census area assignment. Finally, we developed surfaces of spatial patterns in errors in order to identify locations in the study area where exposures may be over-/under-estimated. Our observations provide insights towards refining geocoding methods for epidemiology, and suggest methods for quantifying and interpreting geocoding error with respect to exposure misclassification, towards understanding potential impacts on health effect estimates.

Dosimetric impact of rotational errors on the quality of VMAT-SRS for multiple brain metastases: Comparison between single- and two-isocenter treatment planning techniques

Purpose

In the absence of a 6D couch and/or assuming considerable intrafractional patient motion, rotational errors could affect target coverage and OAR‐sparing especially in multiple metastases VMAT‐SRS cranial cases, which often involve the concurrent irradiation of off‐axis targets. This work aims to study the dosimetric impact of rotational errors in such applications, under a comparative perspective between the single‐ and two‐isocenter treatment techniques.

Methods

Ten patients (36 metastases) were included in this study. Challenging cases were only considered, with several targets lying in close proximity to OARs. Two multiarc VMAT plans per patient were prepared, involving one and two isocenters, serving as the reference plans. Different degrees of angular offsets at various orientations were introduced, simulating rotational errors. Resulting dose distributions were evaluated and compared using commonly employed dose‐volume and plan quality indices.

Results

For single‐isocenter plans and 1^⁰ rotations, plan quality indices, such as coverage, conformity index and D_95%, deteriorated significantly (>5%) for distant targets from the isocenter (at> 4–6 cm). Contrarily, for two‐isocenter plans, target distances to nearest isocenter were always shorter (≤4 cm), and, consequently, 1^⁰ errors were well‐tolerated. In the most extreme case considered (2^⁰ around all axes) conformity index deteriorated by on‐average 7.2%/cm of distance to isocenter, if one isocenter is used, and 2.6%/cm, for plans involving two isocenters. The effect is, however, strongly associated with target volume. Regarding OARs, for single‐isocenter plans, significant increase (up to 63%) in D_max and D_0.02cc values was observed for any angle of rotation. Plans that could be considered clinically unacceptable were obtained even for the smallest angle considered, although rarer for the two‐isocenter planning approach.

Conclusion

Limiting the lesion‐to‐isocenter distance to ≤4 cm by introducing additional isocenter(s) appears to partly mitigate severe target underdosage, especially for smaller target sizes. If OAR‐sparing is also a concern, more stringent rotational error tolerances apply.

MRI-Related Geometric Distortions in Stereotactic Radiotherapy Treatment Planning: Evaluation and Dosimetric Impact

In view of their superior soft tissue contrast compared to computed tomography, magnetic resonance images are commonly involved in stereotactic radiosurgery/radiotherapy applications for target delineation purposes. It is known, however, that magnetic resonance images are geometrically distorted, thus deteriorating dose delivery accuracy. The present work focuses on the assessment of geometric distortion inherent in magnetic resonance images used in stereotactic radiosurgery/radiotherapy treatment planning and attempts to quantitively evaluate the consequent impact on dose delivery. The geometric distortions for 3 clinical magnetic resonance protocols (at both 1.5 and 3.0 T) used for stereotactic radiosurgery/radiotherapy treatment planning were evaluated using a recently proposed phantom and methodology. Areas of increased distortion were identified at the edges of the imaged volume which was comparable to a brain scan. Although mean absolute distortion did not exceed 0.5 mm on any spatial axis, maximum detected control point disposition reached 2 mm. In an effort to establish what could be considered as acceptable geometric uncertainty, highly conformal plans were utilized to irradiate targets of different diameters (5-50 mm). The targets were mispositioned by 0.5 up to 3 mm, and dose–volume histograms and plan quality indices clinically used for plan evaluation and acceptance were derived and used to investigate the effect of geometrical uncertainty (distortion) on dose delivery accuracy and plan quality. The latter was found to be strongly dependent on target size. For targets less than 20 mm in diameter, a spatial disposition of the order of 1 mm could significantly affect (>5%) plan acceptance/quality indices. For targets with diameter greater than 2 cm, the corresponding disposition was found greater than 1.5 mm. Overall results of this work suggest that efficacy of stereotactic radiosurgery/radiotherapy applications could be compromised in case of very small targets lying distant from the scanner’s isocenter (eg, the periphery of the brain).

The Neuroergonomics of Vigilance

OBJECTIVE

The aim of this study was to examine the effects of uncertainty about where in the field of view critical signals for detection appear during a vigilance task (spatial uncertainty) on cerebral blood flow velocity (CBFV) and oculomotor fatigue.

BACKGROUND

Neuroergonomics is a dimension of human factors founded by Raja Parasuraman that studies brain functions underlying performance at work. Neuroergonomic studies have shown that observers in vigilance tasks lose information-processing resources over time and experience oculomotor fatigue as indexed by a temporal decline in CBFV and elevation in eye closure as reflected in the PERCLOS metric. Because spatial uncertainty increases an observer's need for visual scanning relative to a spatial certainty condition, it was anticipated that spatial uncertainty would result in a greater temporal decline in CBFV and increased eye closure in a vigilance session.

METHOD

Observers performed a simulated unmanned aerial vehicle (UAV) control task wherein collision flight paths were the events to be detected. UAV images could appear at random in any one of five locations on the controller's display (spatial uncertainty) or only in a fixed location (spatial certainty).

RESULTS

Signal detection was poorer in the spatial-uncertain relative to the certain condition, and predictions regarding CBFV and eye closure were confirmed.

CONCLUSION

Vigilance tasks involving spatial uncertainty are more neurophysiologically taxing than those in which spatial uncertainty is not a factor.

APPLICATION

The neuroergonomic approach helps in understanding the effects of psychophysical factors in vigilance and to signify when performance aiding is needed.

The Impact of Uncertain Contact Location on Situation Awareness and Performance in Simulated Submarine Track Management

OBJECTIVE

The aim of these studies was to examine the extent to which uncertainty in contact location in submarine track management affected operator situation awareness (SA), workload, and performance and whether operator SA predicted unique variance in performance.

BACKGROUND

We extend prior research by manipulating uncertainty in contact location and by including a sample of expert track managers in a submarine combat system.

METHOD

In Experiment 1, university students completed a track management task. In Experiment 2, expert submariners were embedded in a real submarine combat system. Uncertainty was manipulated and SA was measured using the situation present assessment method.

RESULTS

Increased uncertainty led to higher student workload and moderately impaired SA and performance, and SA predicted incremental variance in performance. Uncertainty had no effect on expert SA or the accuracy of the tactical picture compiled. On average, experts took 20 s to accept SA queries (compared with 2.18 s for students). The time taken for experts to accept SA queries, but not their subsequent response to SA queries, was positively associated with their tactical picture accuracy.

CONCLUSION

Uncertainty can negatively impact SA, workload, and performance. Some key findings from the laboratory were replicated using experts, but the fact that experts took on average 20 s to accept SA queries presents a challenge for using SPAM in submarine control rooms.

APPLICATION

Contact location is uncertain due to the use of passive sonar and hostile deception. It is essential to measure track manager SA in order to inform work design and training.

Identifying the Uncertainty in Physician Practice Location through Spatial Analytics and Text Mining

In response to the widespread concern about the adequacy, distribution, and disparity of access to a health care workforce, the correct identification of physicians’ practice locations is critical to access public health services. In prior literature, little effort has been made to detect and resolve the uncertainty about whether the address provided by a physician in the survey is a practice address or a home address. This paper introduces how to identify the uncertainty in a physician’s practice location through spatial analytics, text mining, and visual examination. While land use and zoning code, embedded within the parcel datasets, help to differentiate resident areas from other types, spatial analytics may have certain limitations in matching and comparing physician and parcel datasets with different uncertainty issues, which may lead to unforeseen results. Handling and matching the string components between physicians’ addresses and the addresses of the parcels could identify the spatial uncertainty and instability to derive a more reasonable relationship between different datasets. Visual analytics and examination further help to clarify the undetectable patterns. This research will have a broader impact over federal and state initiatives and policies to address both insufficiency and maldistribution of a health care workforce to improve the accessibility to public health services.

Characterizing spatial uncertainty when integrating social data in conservation planning

Recent conservation planning studies have presented approaches for integrating spatially referenced social (SRS) data with a view to improving the feasibility of conservation action. We reviewed the growing conservation literature on SRS data, focusing on elicited or stated preferences derived through social survey methods such as choice experiments and public participation geographic information systems. Elicited SRS data includes the spatial distribution of willingness to sell, willingness to pay, willingness to act, and assessments of social and cultural values. We developed a typology for assessing elicited SRS data uncertainty which describes how social survey uncertainty propagates when projected spatially and the importance of accounting for spatial uncertainty such as scale effects and data quality. These uncertainties will propagate when elicited SRS data is integrated with biophysical data for conservation planning and may have important consequences for assessing the feasibility of conservation actions. To explore this issue further, we conducted a systematic review of the elicited SRS data literature. We found that social survey uncertainty was commonly tested for, but that these uncertainties were ignored when projected spatially. Based on these results we developed a framework which will help researchers and practitioners estimate social survey uncertainty and use these quantitative estimates to systematically address uncertainty within an analysis. This is important when using SRS data in conservation applications because decisions need to be made irrespective of data quality and well characterized uncertainty can be incorporated into decision theoretic approaches.

Optimal configurations of spatial scale for grid cell firing under noise and uncertainty

We examined the accuracy with which the location of an agent moving within an environment could be decoded from the simulated firing of systems of grid cells. Grid cells were modelled with Poisson spiking dynamics and organized into multiple ‘modules’ of cells, with firing patterns of similar spatial scale within modules and a wide range of spatial scales across modules. The number of grid cells per module, the spatial scaling factor between modules and the size of the environment were varied. Errors in decoded location can take two forms: small errors of precision and larger errors resulting from ambiguity in decoding periodic firing patterns. With enough cells per module (e.g. eight modules of 100 cells each) grid systems are highly robust to ambiguity errors, even over ranges much larger than the largest grid scale (e.g. over a 500 m range when the maximum grid scale is 264 cm). Results did not depend strongly on the precise organization of scales across modules (geometric, co-prime or random). However, independent spatial noise across modules, which would occur if modules receive independent spatial inputs and might increase with spatial uncertainty, dramatically degrades the performance of the grid system. This effect of spatial uncertainty can be mitigated by uniform expansion of grid scales. Thus, in the realistic regimes simulated here, the optimal overall scale for a grid system represents a trade-off between minimizing spatial uncertainty (requiring large scales) and maximizing precision (requiring small scales). Within this view, the temporary expansion of grid scales observed in novel environments may be an optimal response to increased spatial uncertainty induced by the unfamiliarity of the available spatial cues.

Intrasaccadic suppression is dominated by reduced detector gain

Human vision requires fast eye movements (saccades). Each saccade causes a self-induced motion signal, but we are not aware of this potentially jarring visual input. Among the theorized causes of this phenomenon is a decrease in visual sensitivity before (presaccadic suppression) and during (intrasaccadic suppression) saccades. We investigated intrasaccadic suppression using a perceptual template model (PTM) relating visual detection to different signal-processing stages. One stage changes the gain on the detector's input; another increases uncertainty about the stimulus, allowing more noise into the detector; and other stages inject noise into the detector in a stimulus-dependent or -independent manner. By quantifying intrasaccadic suppression of flashed horizontal gratings at varying external noise levels, we obtained threshold-versus-noise (TVN) data, allowing us to fit the PTM. We tested if any of the PTM parameters changed significantly between the fixation and saccade models and could therefore account for intrasaccadic suppression. We found that the dominant contribution to intrasaccadic suppression was a reduction in the gain of the visual detector. We discuss how our study differs from previous ones that have pointed to uncertainty as an underlying cause of intrasaccadic suppression and how the equivalent noise approach provides a framework for comparing the disparate neural correlates of saccadic suppression.