A Review on Autonomous Vehicles: Progress, Methods and Challenges

CBAM VGG16: An efficient driver distraction classification using CBAM embedded VGG16 architecture

Driver monitoring systems (DMS) are crucial in autonomous driving systems (ADS) when users are concerned about driver/vehicle safety. In DMS, the significant influencing factor of driver/vehicle safety is the classification of driver distractions or activities. The driver's distractions or activities convey meaningful information to the ADS, enhancing the driver/ vehicle safety in real-time vehicle driving. The classification of driver distraction or activity is challenging due to the unpredictable nature of human driving. This paper proposes a convolutional block attention module embedded in Visual Geometry Group (CBAM VGG16) deep learning architecture to improve the classification performance of driver distractions. The proposed CBAM VGG16 architecture is the hybrid network of the CBAM layer with conventional VGG16 network layers. Adding a CBAM layer into a traditional VGG16 architecture enhances the model's feature extraction capacity and improves the driver distraction classification results. To validate the significant performance of our proposed CBAM VGG16 architecture, we tested our model on the American University in Cairo (AUC) distracted driver dataset version 2 (AUCD2) for cameras 1 and 2 images. Our experiment results show that the proposed CBAM VGG16 architecture achieved 98.65% classification accuracy for camera 1 and 97.85% for camera 2 AUCD2 datasets. The CBAM VGG16 architecture also compared the driver distraction classification performance with DenseNet121, Xception, MoblieNetV2, InceptionV3, and VGG16 architectures based on the proposed model's accuracy, loss, precision, F1 score, recall, and confusion matrix. The drivers' distraction classification results indicate that the proposed CBAM VGG16 has 3.7% classification improvements for AUCD2 camera 1 images and 5% for camera 2 images compared to the conventional VGG16 deep learning classification model. We also tested our proposed architecture with different hyperparameter values and estimated the optimal values for best driver distraction classification. The significance of data augmentation techniques for the data diversity performance of the CBAM VGG16 model is also validated in terms of overfitting scenarios. The Grad-CAM visualization of our proposed CBAM VGG16 architecture is also considered in our study, and the results show that VGG16 architecture without CBAM layers is less attentive to the essential parts of the driver distraction images. Furthermore, we tested the effective classification performance of our proposed CBAM VGG16 architecture with the number of model parameters, model size, various input image resolutions, cross-validation, Bayesian search optimization and different CBAM layers. The results indicate that CBAM layers in our proposed architecture enhance the classification performance of conventional VGG16 architecture and outperform the state-of-the-art deep learning architectures.

DriveLLaVA: Human-Level Behavior Decisions via Vision Language Model

Human-level driving is the ultimate goal of autonomous driving. As the top-level decision-making aspect of autonomous driving, behavior decision establishes short-term driving behavior strategies by evaluating road structures, adhering to traffic rules, and analyzing the intentions of other traffic participants. Existing behavior decisions are primarily implemented based on rule-based methods, exhibiting insufficient generalization capabilities when faced with new and unseen driving scenarios. In this paper, we propose a novel behavior decision method that leverages the inherent generalization and commonsense reasoning abilities of visual language models (VLMs) to learn and simulate the behavior decision process in human driving. We constructed a novel instruction-following dataset containing a large number of image-text instructions paired with corresponding driving behavior labels, to support the learning of the Drive Large Language and Vision Assistant (DriveLLaVA) and enhance the transparency and interpretability of the entire decision process. DriveLLaVA is fine-tuned on this dataset using the Low-Rank Adaptation (LoRA) approach, which efficiently optimizes the model parameter count and significantly reduces training costs. We conducted extensive experiments on a large-scale instruction-following dataset, and compared with state-of-the-art methods, DriveLLaVA demonstrated excellent behavior decision performance. DriveLLaVA is capable of handling various complex driving scenarios, showing strong robustness and generalization abilities.

A matched case-control analysis of autonomous vs human-driven vehicle accidents

Despite the recent advancements that Autonomous Vehicles have shown in their potential to improve safety and operation, considering differences between Autonomous Vehicles and Human-Driven Vehicles in accidents remain unidentified due to the scarcity of real-world Autonomous Vehicles accident data. We investigated the difference in accident occurrence between Autonomous Vehicles' levels and Human-Driven Vehicles by utilizing 2100 Advanced Driving Systems and Advanced Driver Assistance Systems and 35,113 Human-Driven Vehicles accident data. A matched case-control design was conducted to investigate the differential characteristics involving Autonomous' versus Human-Driven Vehicles' accidents. The analysis suggests that accidents of vehicles equipped with Advanced Driving Systems generally have a lower chance of occurring than Human-Driven Vehicles in most of the similar accident scenarios. However, accidents involving Advanced Driving Systems occur more frequently than Human-Driven Vehicle accidents under dawn/dusk or turning conditions, which is 5.25 and 1.98 times higher, respectively. Our research reveals the accident risk disparities between Autonomous Vehicles and Human-Driven Vehicles, informing future development in Autonomous technology and safety enhancements.

Responsiveness and Precision of Digital IMUs under Linear and Curvilinear Motion Conditions for Local Navigation and Positioning in Advanced Smart Mobility

Sensors based on MEMS technology, in particular Inertial Measurement Units (IMUs), when installed on vehicles, provide a real-time full estimation of vehicles' state vector (e.g., position, velocity, yaw angle, angular rate, acceleration), which is required for the planning and control of cars' trajectories, as well as managing the in-car local navigation and positioning tasks. Moreover, data provided by the IMUs, integrated with the data of multiple inputs from other sensing systems (such as Lidar, cameras, and GPS) within the vehicle, and with the surrounding information exchanged in real time (vehicle to vehicle, vehicle to infrastructure, or vehicle to other entities), can be exploited to actualize the full implementation of "smart mobility" on a large scale. On the other hand, "smart mobility" (which is expected to improve road safety, reduce traffic congestion and environmental burden, and enhance the sustainability of mobility as a whole), to be safe and functional on a large scale, should be supported by highly accurate and trustworthy technologies based on precise and reliable sensors and systems. It is known that the accuracy and precision of data supplied by appropriately in-lab-calibrated IMUs (with respect to the primary or secondary standard in order to provide traceability to the International System of Units) allow guaranteeing high quality, reliable information managed by processing systems, since they are reproducible, repeatable, and traceable. In this work, the effective responsiveness and the related precision of digital IMUs, under sinusoidal linear and curvilinear motion conditions at 5 Hz, 10 Hz, and 20 Hz, are investigated on the basis of metrological approaches in laboratory standard conditions only. As a first step, in-lab calibrations allow one to reduce the variables of uncontrolled boundary conditions (e.g., occurring in vehicles in on-site tests) in order to identify the IMUs' sensitivity in a stable and reproducible environment. For this purpose, a new calibration system, based on an oscillating rotating table was developed to reproduce the dynamic conditions of use in the field, and the results are compared with calibration data obtained on linear calibration benches.

A precautionary approach to autonomous vehicles

In this article, we defend an approach to autonomous vehicle ethics and policy based on the precautionary principle. We argue that a precautionary approach is warranted, given the significant scientific and moral uncertainties related to autonomous vehicles, especially higher-level ones. While higher-level autonomous vehicles may offer many important benefits to society, they also pose significant risks, which are not fully understood at this juncture. Risk management strategies traditionally used by government officials to make decisions about new technologies cannot be applied to higher-level autonomous vehicles because these strategies require accurate and reliable probability estimates concerning the outcomes of different policy options and extensive agreement about values, which are not currently available for autonomous vehicles. Although we describe our approach as precautionary, that does not mean that we are opposed to autonomous vehicle development and deployment, because autonomous vehicles offer benefits that should be pursued. The optimal approach to managing the risks of autonomous vehicles is to take reasonable precautions; that is, to adopt policies that attempt to deal with serious risks in a responsible way without depriving society of important benefits.

Design, Construction, and Validation of an Experimental Electric Vehicle with Trajectory Tracking

This research presents an experimental electric vehicle developed at the Tecnológico Nacional de México Celaya campus. It was decided to use a golf cart-type gasoline vehicle as a starting point. Initially, the body was removed, and the vehicle was electrified, meaning its engine was replaced with an electric one. Subsequently, sensors used to measure the vehicle states were placed, calibrated, and instrumented. Additionally, a mathematical model was developed along with a strategy for the parametric identification of this model. A communication scheme was implemented consisting of four slave devices responsible for controlling the accelerator, brake, steering wheel, and measuring the sensors related to odometry. The master device is responsible for communicating with the slaves, displaying information on a screen, creating a log, and implementing trajectory tracking techniques based on classical, geometric, and predictive control. Finally, the performance of the control algorithms implemented on the experimental prototype was compared in terms of tracking error and control input across three different types of trajectories: lane change, right-angle curve, and U-turn.

Learning-Based Hierarchical Decision-Making Framework for Automatic Driving in Incompletely Connected Traffic Scenarios

The decision-making algorithm serves as a fundamental component for advancing the level of autonomous driving. The end-to-end decision-making algorithm has a strong ability to process the original data, but it has grave uncertainty. However, other learning-based decision-making algorithms rely heavily on ideal state information and are entirely unsuitable for autonomous driving tasks in real-world scenarios with incomplete global information. Addressing this research gap, this paper proposes a stable hierarchical decision-making framework with images as the input. The first step of the framework is a model-based data encoder that converts the input image data into a fixed universal data format. Next is a state machine based on a time series Graph Convolutional Network (GCN), which is used to classify the current driving state. Finally, according to the state's classification, the corresponding rule-based algorithm is selected for action generation. Through verification, the algorithm demonstrates the ability to perform autonomous driving tasks in different traffic scenarios without relying on global network information. Comparative experiments further confirm the effectiveness of the hierarchical framework, model-based image data encoder, and time series GCN.

OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning

Federated learning (FL) has emerged as a promising approach to collaboratively train machine learning models across multiple edge devices while preserving privacy. The success of FL hinges on the efficiency of participating models and their ability to handle the unique challenges of distributed learning. While several variants of Vision Transformer (ViT) have shown great potential as alternatives to modern convolutional neural networks (CNNs) for centralized training, the unprecedented size and higher computational demands hinder their deployment on resource-constrained edge devices, challenging their widespread application in FL. Since client devices in FL typically have limited computing resources and communication bandwidth, models intended for such devices must strike a balance between model size, computational efficiency, and the ability to adapt to the diverse and non-IID data distributions encountered in FL. To address these challenges, we propose OnDev-LCT: Lightweight Convolutional Transformers for On-Device vision tasks with limited training data and resources. Our models incorporate image-specific inductive biases through the LCT tokenizer by leveraging efficient depthwise separable convolutions in residual linear bottleneck blocks to extract local features, while the multi-head self-attention (MHSA) mechanism in the LCT encoder implicitly facilitates capturing global representations of images. Extensive experiments on benchmark image datasets indicate that our models outperform existing lightweight vision models while having fewer parameters and lower computational demands, making them suitable for FL scenarios with data heterogeneity and communication bottlenecks.

Road-MobileSeg: Lightweight and Accurate Road Extraction Model from Remote Sensing Images for Mobile Devices

Current road extraction models from remote sensing images based on deep learning are computationally demanding and memory-intensive because of their high model complexity, making them impractical for mobile devices. This study aimed to develop a lightweight and accurate road extraction model, called Road-MobileSeg, to address the problem of automatically extracting roads from remote sensing images on mobile devices. The Road-MobileFormer was designed as the backbone structure of Road-MobileSeg. In the Road-MobileFormer, the Coordinate Attention Module was incorporated to encode both channel relationships and long-range dependencies with precise position information for the purpose of enhancing the accuracy of road extraction. Additionally, the Micro Token Pyramid Module was introduced to decrease the number of parameters and computations required by the model, rendering it more lightweight. Moreover, three model structures, namely Road-MobileSeg-Tiny, Road-MobileSeg-Small, and Road-MobileSeg-Base, which share a common foundational structure but differ in the quantity of parameters and computations, were developed. These models varied in complexity and were available for use on mobile devices with different memory capacities and computing power. The experimental results demonstrate that the proposed models outperform the compared typical models in terms of accuracy, lightweight structure, and latency and achieve high accuracy and low latency on mobile devices. This indicates that the models that integrate with the Coordinate Attention Module and the Micro Token Pyramid Module surpass the limitations of current research and are suitable for road extraction from remote sensing images on mobile devices.

Leveraging LiDAR-Based Simulations to Quantify the Complexity of the Static Environment for Autonomous Vehicles in Rural Settings

This paper uses virtual simulations to examine the interaction between autonomous vehicles (AVs) and their surrounding environment. A framework was developed to estimate the environment's complexity by calculating the real-time data processing requirements for AVs to navigate effectively. The VISTA simulator was used to synthesize viewpoints to replicate the captured environment accurately. With an emphasis on static physical features, roadways were dissected into relevant road features (RRFs) and full environment (FE) to study the impact of roadside features on the scene complexity and demonstrate the gravity of wildlife-vehicle collisions (WVCs) on AVs. The results indicate that roadside features substantially increase environmental complexity by up to 400%. Increasing a single lane to the road was observed to increase the processing requirements by 12.3-16.5%. Crest vertical curves decrease data rates due to occlusion challenges, with a reported average of 4.2% data loss, while sag curves can increase the complexity by 7%. In horizontal curves, roadside occlusion contributed to severe loss in road information, leading to a decrease in data rate requirements by as much as 19%. As for weather conditions, heavy rain increased the AV's processing demands by a staggering 240% when compared to normal weather conditions. AV developers and government agencies can exploit the findings of this study to better tailor AV designs and meet the necessary infrastructure requirements.

Intelligent Vehicle Decision-Making and Trajectory Planning Method Based on Deep Reinforcement Learning in the Frenet Space

The complexity inherent in navigating intricate traffic environments poses substantial hurdles for intelligent driving technology. The continual progress in mapping and sensor technologies has equipped vehicles with the capability to intricately perceive their exact position and the intricate interplay among surrounding traffic elements. Building upon this foundation, this paper introduces a deep reinforcement learning method to solve the decision-making and trajectory planning problem of intelligent vehicles. The method employs a deep learning framework for feature extraction, utilizing a grid map generated from a blend of static environmental markers such as road centerlines and lane demarcations, in addition to dynamic environmental cues including vehicle positions across varied lanes, all harmonized within the Frenet coordinate system. The grid map serves as the input for the state space, and the input for the action space comprises a vector encompassing lane change timing, velocity, and vertical displacement at the lane change endpoint. To optimize the action strategy, a reinforcement learning approach is employed. The feasibility, stability, and efficiency of the proposed method are substantiated via experiments conducted in the CARLA simulator across diverse driving scenarios, and the proposed method can increase the average success rate of lane change by 6.8% and 13.1% compared with the traditional planning control algorithm and the simple reinforcement learning method.

Vehicle Detection and Tracking with Roadside LiDAR Using Improved ResNet18 and the Hungarian Algorithm

By the end of the 2020s, full autonomy in autonomous driving may become commercially viable in certain regions. However, achieving Level 5 autonomy requires crucial collaborations between vehicles and infrastructure, necessitating high-speed data processing and low-latency capabilities. This paper introduces a vehicle tracking algorithm based on roadside LiDAR (light detection and ranging) infrastructure to reduce the latency to 100 ms without compromising the detection accuracy. We first develop a vehicle detection architecture based on ResNet18 that can more effectively detect vehicles at a full frame rate by improving the BEV mapping and the loss function of the optimizer. Then, we propose a new three-stage vehicle tracking algorithm. This algorithm enhances the Hungarian algorithm to better match objects detected in consecutive frames, while time-space logicality and trajectory similarity are proposed to address the short-term occlusion problem. Finally, the system is tested on static scenes in the KITTI dataset and the MATLAB/Simulink simulation dataset. The results show that the proposed framework outperforms other methods, with F1-scores of 96.97% and 98.58% for vehicle detection for the KITTI and MATLAB/Simulink datasets, respectively. For vehicle tracking, the MOTA are 88.12% and 90.56%, and the ID-F1 are 95.16% and 96.43%, which are better optimized than the traditional Hungarian algorithm. In particular, it has a significant improvement in calculation speed, which is important for real-time transportation applications.

Cybersecurity in Internet of Medical Vehicles: State-of-the-Art Analysis, Research Challenges and Future Perspectives

The "Internet-of-Medical-Vehicles (IOMV)" is one of the special applications of the Internet of Things resulting from combining connected healthcare and connected vehicles. As the IOMV communicates with a variety of networks along its travel path, it incurs various security risks due to sophisticated cyber-attacks. This can endanger the onboard patient's life. So, it is critical to understand subjects related to "cybersecurity" in the IOMV to develop robust cybersecurity measures. In this paper, the goal is to evaluate recent trends and state-of-the-art publications, gaps, and future outlooks related to this research area. With this aim, a variety of publications between 2016 and 2023 from "Web-of-Science" and "Scopus" databases were analysed. Our analysis revealed that the IOMV is a niche and unexplored research area with few defined standards and frameworks, and there is a great need to implement robust cybersecurity measures. This paper will help researchers to gain a comprehensive idea of this niche research topic, as it presents an analysis of top journals and highly cited papers, their challenges and limitations, the system model and architecture of the IOMV, related applicable standards, potential cyber-attacks, factors causing cybersecurity risks, various artificial intelligence techniques for developing potential countermeasures, the assessment and parameterisation of cybersecurity risks, constraints and challenges, and future outlooks for implementing cybersecurity measures in the IOMV.

Autonomous Robots for Services-State of the Art, Challenges, and Research Areas

It has been almost half a century since the first interest in autonomous robots was shown, and research is still continuing to improve their ability to make perfectly conscious decisions from a user safety point of view. These autonomous robots are now at a fairly advanced level, which means that their adoption rate in social environments is also increasing. This article reviews the current state of development of this technology and highlights the evolution of interest in it. We analyze and discuss specific areas of its use, for example, its functionality and current level of development. Finally, challenges related to the current level of research and new methods that are still being developed for the wider adoption of these autonomous robots are highlighted.

A Tube Model Predictive Control Method for Autonomous Lateral Vehicle Control Based on Sliding Mode Control

This paper aims to enhance the lateral path tracking control of autonomous vehicles (AV) in the presence of external disturbances. While AV technology has made significant strides, real-world driving scenarios often pose challenges such as slippery or uneven roads, which can adversely affect the lateral path tracking control and reduce driving safety and efficiency. Conventional control algorithms struggle to address this issue due to their inability to account for unmodeled uncertainties and external disturbances. To tackle this problem, this paper proposes a novel algorithm that combines robust sliding mode control (SMC) and tube model predictive control (MPC). The proposed algorithm leverages the strengths of both MPC and SMC. Specifically, MPC is used to derive the control law for the nominal system to track the desired trajectory. The error system is then employed to minimize the difference between the actual state and the nominal state. Finally, the sliding surface and reaching law of SMC are utilized to derive an auxiliary tube SMC control law, which helps the actual system keep up with the nominal system and achieve robustness. Experimental results demonstrate that the proposed method outperforms conventional tube MPC, linear quadratic regulator (LQR) algorithms, and MPC in terms of robustness and tracking accuracy, especially in the presence of unmodeled uncertainties and external disturbances.

A Spatiotemporal Calibration Algorithm for IMU-LiDAR Navigation System Based on Similarity of Motion Trajectories

The fusion of light detection and ranging (LiDAR) and inertial measurement unit (IMU) sensing information can effectively improve the environment modeling and localization accuracy of navigation systems. To realize the spatiotemporal unification of data collected by the IMU and the LiDAR, a two-step spatiotemporal calibration method combining coarse and fine is proposed. The method mainly includes two aspects: (1) Modeling continuous-time trajectories of IMU attitude motion using B-spline basis functions; the motion of the LiDAR is estimated by using the normal distributions transform (NDT) point cloud registration algorithm, taking the Hausdorff distance between the local trajectories as the cost function and combining it with the hand-eye calibration method to solve the initial value of the spatiotemporal relationship between the two sensors' coordinate systems, and then using the measurement data of the IMU to correct the LiDAR distortion. (2) According to the IMU preintegration, and the point, line, and plane features of the lidar point cloud, the corresponding nonlinear optimization objective function is constructed. Combined with the corrected LiDAR data and the initial value of the spatiotemporal calibration of the coordinate systems, the target is optimized under the nonlinear graph optimization framework. The rationality, accuracy, and robustness of the proposed algorithm are verified by simulation analysis and actual test experiments. The results show that the accuracy of the proposed algorithm in the spatial coordinate system relationship calibration was better than 0.08° (3δ) and 5 mm (3δ), respectively, and the time deviation calibration accuracy was better than 0.1 ms and had strong environmental adaptability. This can meet the high-precision calibration requirements of multisensor spatiotemporal parameters of field robot navigation systems.