Towards High-Definition 3D Urban Mapping: Road Feature-Based Registration of Mobile Mapping Systems and Aerial Imagery

Traffic lights detection and tracking for HD map creation

HD-maps are one of the core components of the self-driving pipeline. Despite the effort of many companies to develop a completely independent vehicle, many state-of-the-art solutions rely on high-definition maps of the environment for localization and navigation. Nevertheless, the creation process of such maps can be complex and error-prone or expensive if performed via ad-hoc surveys. For this reason, robust automated solutions are required. One fundamental component of an high-definition map is traffic lights. In particular, traffic light detection has been a well-known problem in the autonomous driving field. Still, the focus has always been on the light state, not the features (i.e., shape, orientation, pictogram). This work presents a pipeline for lights HD-map creation designed to provide accurate georeferenced position and description of all traffic lights seen by a camera mounted on a surveying vehicle. Our algorithm considers consecutive detection of the same light and uses Kalman filtering techniques to provide each target's smoother and more precise position. Our pipeline has been validated for the detection and mapping task using the state-of-the-art dataset DriveU Traffic Light Dataset. The results show that our model is robust even with noisy GPS data. Moreover, for the detection task, we highlight how our model can correctly identify even far-away targets which are not labeled in the original dataset.

Design and Implementation of HD Mapping, Vehicle Control, and V2I Communication for Robo-Taxi Services

This paper presents our autonomous driving (AD) software stack, developed to complete the main mission of the contest we entered. The main mission can be simply described as a robo-taxi service on public roads, to transport passengers to their destination autonomously. Among the key competencies required for the main mission, this paper focused on high-definition mapping, vehicle control, and vehicle-to-infrastructure (V2I) communication. V2I communication refers to the task of wireless data exchange between a roadside unit and vehicles. With the data being captured and shared, rich, timely, and non-line-of-sight-aware traffic information can be utilized for a wide range of AD applications. In the contest, V2I communication was applied for a robo-taxi service, and also for traffic light recognition. A V2I communication-enabled traffic light recognizer was developed, to achieve a nearly perfect recognition rate, and a robo-taxi service handler was developed, to perform the main mission of the contest.

Analysis of Occlusion Effects for Map-Based Self-Localization in Urban Areas

A high-definition (HD) map provides structural information for map-based self-localization, enabling stable estimation in real environments. In urban areas, there are many obstacles, such as buses, that occlude sensor observations, resulting in self-localization errors. However, most of the existing HD map-based self-localization evaluations do not consider sudden significant errors due to obstacles. Instead, they evaluate this in terms of average error over estimated trajectories in an environment with few occlusions. This study evaluated the effects of self-localization estimation on occlusion with synthetically generated obstacles in a real environment. Various patterns of synthetic occlusion enabled the analyses of the effects of self-localization error from various angles. Our experiments showed various characteristics that locations susceptible to obstacles have. For example, we found that occlusion in intersections tends to increase self-localization errors. In addition, we analyzed the geometrical structures of a surrounding environment in high-level error cases and low-level error cases with occlusions. As a result, we suggested the concept that the real environment should have to achieve robust self-localization under occlusion conditions.

Nighttime Mobile Laser Scanning and 3D Luminance Measurement: Verifying the Outcome of Roadside Tree Pruning with Mobile Measurement of the Road Environment

Roadside vegetation can affect the performance of installed road lighting. We demonstrate a workflow in which a car-mounted measurement system is used to assess the light-obstructing effect of roadside vegetation. The mobile mapping system (MMS) includes a panoramic camera system, laser scanner, inertial measurement unit, and satellite positioning system. The workflow and the measurement system were applied to a road section of Munkkiniemenranta, Helsinki, Finland, in 2015 and 2019. The relative luminance distribution on a road surface and the obstructing vegetation were measured before and after roadside vegetation pruning applying a luminance-calibrated mobile mapping system. The difference between the two measurements is presented, and the opportunities provided by the mobile 3D luminance measurement system are discussed.

Automated Geolocation in Urban Environments Using a Simple Camera-Equipped Unmanned Aerial Vehicle: A Rapid Mapping Surveying Alternative?

GNSS positioning accuracy can be degraded in areas where the surrounding object geometry and morphology interacts with the GNSS signals. Specifically, urban environments pose challenges to precise GNSS positioning because of signal interference or interruptions. Also, non-GNSS surveying methods, including total stations and laser scanners, involve time consuming practices in the field and costly equipment. The present study proposes the use of an Unmanned Aerial Vehicle (UAV) for autonomous rapid mapping that resolves the problem of localization for the drone itself by acquiring location information of characteristic points on the ground in a local coordinate system using simultaneous localization and mapping (SLAM) and vision algorithms. A common UAV equipped with a camera and at least a single known point, are enough to produce a local map of the scene and to estimate the relative coordinates of pre-defined ground points along with an additional arbitrary point cloud. The resulting point cloud is readily measurable for extracting and interpreting geometric information from the area of interest. Under two novel optimization procedures performing line and plane alignment of the UAV-camera-measured point geometries, a set of experiments determines that the localization of a visual point in distances reaching 15 m from the origin, delivered a level of accuracy under 50 cm. Thus, a simple UAV with an optical sensor and a visual marker, prove quite promising and cost-effective for rapid mapping and point localization in an unknown environment.

Estimating Autonomous Vehicle Localization Error Using 2D Geographic Information

Accurately and precisely knowing the location of the vehicle is a critical requirement for safe and successful autonomous driving. Recent studies suggest that error for map-based localization methods are tightly coupled with the surrounding environment. Considering this relationship, it is therefore possible to estimate localization error by quantifying the representation and layout of real-world phenomena. To date, existing work on estimating localization error have been limited to using self-collected 3D point cloud maps. This paper investigates the use of pre-existing 2D geographic information datasets as a proxy to estimate autonomous vehicle localization error. Seven map evaluation factors were defined for 2D geographic information in a vector format, and random forest regression was used to estimate localization error for five experiment paths in Shinjuku, Tokyo. In the best model, the results show that it is possible to estimate autonomous vehicle localization error with 69.8% of predictions within 2.5 cm and 87.4% within 5 cm.

The Auto-Complete Graph: Merging and Mutual Correction of Sensor and Prior Maps for SLAM

Simultaneous Localization And Mapping (SLAM) usually assumes the robot starts without knowledge of the environment. While prior information, such as emergency maps or layout maps, is often available, integration is not trivial since such maps are often out of date and have uncertainty in local scale. Integration of prior map information is further complicated by sensor noise, drift in the measurements, and incorrect scan registrations in the sensor map. We present the Auto-Complete Graph (ACG), a graph-based SLAM method merging elements of sensor and prior maps into one consistent representation. After optimizing the ACG, the sensor map’s errors are corrected thanks to the prior map, while the sensor map corrects the local scale inaccuracies in the prior map. We provide three datasets with associated prior maps: two recorded in campus environments, and one from a fireman training facility. Our method handled up to 40% of noise in odometry, was robust to varying levels of details between the prior and the sensor map, and could correct local scale errors of the prior. In field tests with ACG, users indicated points of interest directly on the prior before exploration. We did not record failures in reaching them.

Towards HD Maps from Aerial Imagery: Robust Lane Marking Segmentation Using Country-Scale Imagery

The upraise of autonomous driving technologies asks for maps characterized bya broad range of features and quality parameters, in contrast to traditional navigation maps which in most cases are enriched graph-based models. This paper tackles several uncertainties within the domain of HD Maps. The authors give an overview about the current state in extracting road features from aerial imagery for creating HD maps, before shifting the focus of the paper towards remote sensing technology. Possible data sources and their relevant parameters are listed. A random forest classifier is used, showing how these data can deliver HD Maps on a country-scale, meeting specific quality parameters.

Registration of Aerial Optical Images with LiDAR Data Using the Closest Point Principle and Collinearity Equations

Registration of large-scale optical images with airborne LiDAR data is the basis of the integration of photogrammetry and LiDAR. However, geometric misalignments still exist between some aerial optical images and airborne LiDAR point clouds. To eliminate such misalignments, we extended a method for registering close-range optical images with terrestrial LiDAR data to a variety of large-scale aerial optical images and airborne LiDAR data. The fundamental principle is to minimize the distances from the photogrammetric matching points to the terrestrial LiDAR data surface. Except for the satisfactory efficiency of about 79 s per 6732 × 8984 image, the experimental results also show that the unit weighted root mean square (RMS) of the image points is able to reach a sub-pixel level (0.45 to 0.62 pixel), and the actual horizontal and vertical accuracy can be greatly improved to a high level of 1/4⁻1/2 (0.17⁻0.27 m) and 1/8⁻1/4 (0.10⁻0.15 m) of the average LiDAR point distance respectively. Finally, the method is proved to be more accurate, feasible, efficient, and practical in variety of large-scale aerial optical image and LiDAR data.