Obstacle Recognition Based on Machine Learning for On-Chip LiDAR Sensors in a Cyber-Physical System

Small obstacle size prediction based on a GA-BP neural network

Accurate and effective acquisition of obstacle size parameters is the basis for environment perception, path planning, and autonomous navigation of mobile robots, and is the key to improve the walking performance of mobile robots. In this paper, a generic algorithm-back propagation (GA-BP) neural network-based method for small obstacle size prediction is proposed for mobile robots to perceive the environment quantitatively. A machine vision-based small obstacle size measurement experiment was designed, and 228 sets of sample data were obtained. A genetic algorithm optimized back propagation neural network was used to build a small obstacle size prediction model with obstacle pixel width, pixel height, pixel area, and obstacle-to-camera distance as input parameters and actual obstacle width, actual height, and actual area as output parameters. The results show that the correlation coefficient (R²) between the predicted and expected values of the test data is higher than 0.999, the root mean square error is lower than 5.573, and the mean absolute percentage error is lower than 2.84%. The good agreement between its predicted and expected values indicates that the model can accurately predict the size of small obstacles. The GA-BP neural network-based small obstacle size prediction method proposed in this paper is simple to execute, has good real-time performance, and provides a new, to the best of our knowledge, way of thinking for mobile robots to acquire environmental data.

Predicting the Traffic Capacity of an Intersection Using Fuzzy Logic and Computer Vision

This paper presents the application of simulation to assess and predict the influence of random factors of pedestrian flow and its continuity on the traffic capacity of a signal-controlled intersection during a right turn. The data were collected from the surveillance cameras of 25 signal-controlled intersections in the city of Chelyabinsk, Russia, and interpreted by a neural network. We considered the influence of both the intensity of the pedestrian flow and its continuity on the traffic capacity of a signal-controlled intersection in the stochastic approach, provided that the flow of vehicles is redundant. We used a reasonably minimized regression model as the basis for our intersection models. At the first stage, we obtained and tested a simulated continuous-stochastic intersection model that accounts for the dynamics of traffic flow. The second approach, due to the unpredictability of pedestrian flow, used a relevant method for analysing traffic flows based on the fuzzy logic theory. The second was also used as the foundation to build and graphically demonstrate a computer model in the fuzzy TECH suite for predictive visualization of the values of a traffic flow crossing a signal-controlled intersection. The results of this study can contribute to understanding the real conditions at a signal-controlled intersection and making grounded decisions on its focused control.

An amalgamation of YOLOv4 and XGBoost for next-gen smart traffic management system

The traffic congestion and the rise in the number of vehicles have become a grievous issue, and it is focused worldwide. One of the issues with traffic management is that the traffic light's timer is not dynamic. As a result, one has to remain longer even if there are no or fewer vehicles, on a roadway, causing unnecessary waiting time, fuel consumption and leads to pollution. Prior work on smart traffic management systems repurposes the use of Internet of things, Time Series Forecasting, and Digital Image Processing. Computer Vision-based smart traffic management is an emerging area of research. Therefore a real-time traffic light optimization algorithm that uses Machine Learning and Deep Learning Techniques to predict the optimal time required by the vehicles to clear the lane is presented. This article concentrates on a two-step approach. The first step is to obtain the count of the independent category of the class of vehicles. For this, the You Only Look Once version 4 (YOLOv4) object detection technique is employed. In the second step, an ensemble technique named eXtreme Gradient Boosting (XGBoost) for predicting the optimal time of the green light window is implemented. Furthermore, the different implemented versions of YOLO and different prediction algorithms are compared with the proposed approach. The experimental analysis signifies that YOLOv4 with the XGBoost algorithm produces the most precise outcomes with a balance of accuracy and inference time. The proposed approach elegantly reduces an average of 32.3% of waiting time with usual traffic on the road.

The Application of an Adaptive Genetic Algorithm Based on Collision Detection in Path Planning of Mobile Robots

An adaptive genetic algorithm based on collision detection (AGACD) is proposed to solve the problems of the basic genetic algorithm in the field of path planning, such as low convergence path quality, many iterations required for convergence, and easily falling into the local optimal solution. First, this paper introduces the Delphi weight method to evaluate the weight of path length, path smoothness, and path safety in the fitness function, and a collision detection method is proposed to detect whether the planned path collides with obstacles. Then, the population initialization process is improved to reduce the program running time. After comprehensively considering the population diversity and the number of algorithm iterations, the traditional crossover operator and mutation operator are improved, and the adaptive crossover operator and adaptive mutation operator are proposed to avoid the local optimal solution. Finally, an optimization operator is proposed to improve the quality of convergent individuals through the second optimization of convergent individuals. The simulation results show that the adaptive genetic algorithm based on collision detection is not only suitable for simulation maps with various sizes and obstacle distributions but also has excellent performance, such as greatly reducing the running time of the algorithm program, and the adaptive genetic algorithm based on collision detection can effectively solve the problems of the basic genetic algorithm.

Variable Selection from Image Texture Feature for Automatic Classification of Concrete Surface Voids

Machine learning plays an important role in computational intelligence and has been widely used in many engineering fields. Surface voids or bugholes frequently appearing on concrete surface after the casting process make the corresponding manual inspection time consuming, costly, labor intensive, and inconsistent. In order to make a better inspection of the concrete surface, automatic classification of concrete bugholes is needed. In this paper, a variable selection strategy is proposed for pursuing feature interpretability, together with an automatic ensemble classification designed for getting a better accuracy of the bughole classification. A texture feature deriving from the Gabor filter and gray-level run lengths is extracted in concrete surface images. Interpretable variables, which are also the components of the feature, are selected according to a presented cumulative voting strategy. An ensemble classifier with its base classifier automatically assigned is provided to detect whether a surface void exists in an image or not. Experimental results on 1000 image samples indicate the effectiveness of our method with a comparable prediction accuracy and model explicable.

Establishment of the Predicting Models of the Dyeing Effect in Supercritical Carbon Dioxide Based on the Generalized Regression Neural Network and Back Propagation Neural Network

With the growing demand of supercritical carbon dioxide (SC-CO2) dyeing, it is important to precisely predict the dyeing effect of supercritical carbon dioxide. In this work, Generalized Regression Neural Network (GRNN) and Back Propagation Neural Network (BPNN) models have been employed to predict the dyeing effect of SC-CO2. These two models have been constructed based on published experimental data and calculated values. A total of 386 experimental data sets were used in the present work. In GRNN and BPNN models, two input parameters, such as temperature, pressure, dye stuff types, carrier types and dyeing time, were selected for the input layer and one variable, K/S value or dye-uptake, was used in the output layer. It was found that the values of mean-relative-error (MRE) for BPNN model and for GRNN model are 3.27–6.54% and 1.68–3.32%, respectively. The results demonstrate that both BPNN and GPNN models can accurately predict the effect of supercritical dyeing but the former is better than the latter.

A Deep Learning Method for Near-Real-Time Cloud and Cloud Shadow Segmentation from Gaofen-1 Images

In this study, an essential application of remote sensing using deep learning functionality is presented. Gaofen-1 satellite mission, developed by the China National Space Administration (CNSA) for the civilian high-definition Earth observation satellite program, provides near-real-time observations for geographical mapping, environment surveying, and climate change monitoring. Cloud and cloud shadow segmentation are a crucial element to enable automatic near-real-time processing of Gaofen-1 images, and therefore, their performances must be accurately validated. In this paper, a robust multiscale segmentation method based on deep learning is proposed to improve the efficiency and effectiveness of cloud and cloud shadow segmentation from Gaofen-1 images. The proposed method first implements feature map based on the spectral-spatial features from residual convolutional layers and the cloud/cloud shadow footprints extraction based on a novel loss function to generate the final footprints. The experimental results using Gaofen-1 images demonstrate the more reasonable accuracy and efficient computational cost achievement of the proposed method compared to the cloud and cloud shadow segmentation performance of two existing state-of-the-art methods.

Computer Vision System for Welding Inspection of Liquefied Petroleum Gas Pressure Vessels Based on Combined Digital Image Processing and Deep Learning Techniques

One of the most important operations during the manufacturing process of a pressure vessel is welding. The result of this operation has a great impact on the vessel integrity; thus, welding inspection procedures must detect defects that could lead to an accident. This paper introduces a computer vision system based on structured light for welding inspection of liquefied petroleum gas (LPG) pressure vessels by using combined digital image processing and deep learning techniques. The inspection procedure applied prior to the welding operation was based on a convolutional neural network (CNN), and it correctly detected the misalignment of the parts to be welded in 97.7% of the cases during the method testing. The post-welding inspection procedure was based on a laser triangulation method, and it estimated the weld bead height and width, with average relative errors of 2.7% and 3.4%, respectively, during the method testing. This post-welding inspection procedure allows us to detect geometrical nonconformities that compromise the weld bead integrity. By using this system, the quality index of the process was improved from 95.0% to 99.5% during practical validation in an industrial environment, demonstrating its robustness.

Machine Learning Methods for Herschel–Bulkley Fluids in Annulus: Pressure Drop Predictions and Algorithm Performance Evaluation

Accurate measurement of pressure drop in energy sectors especially oil and gas exploration is a challenging and crucial parameter for optimization of the extraction process. Many empirical and analytical solutions have been developed to anticipate pressure loss for non-Newtonian fluids in concentric and eccentric pipes. Numerous attempts have been made to extend these models to forecast pressure loss in the annulus. However, there remains a void in the experimental and theoretical studies to establish a model capable of estimating it with higher accuracy and lower computation. Rheology of fluid and geometry of system cumulatively dominate the pressure gradient in an annulus. In the present research, the prediction for Herschel–Bulkley fluids is analyzed by Bayesian Neural Network (BNN), random forest (RF), artificial neural network (ANN), and support vector machines (SVM) for pressure loss in the concentric and eccentric annulus. This study emphasizes on the performance evaluation of given algorithms and their pitfalls in predicting accurate pressure drop. The predictions of BNN and RF exhibit the least mean absolute error of 3.2% and 2.57%, respectively, and both can generalize the pressure loss calculation. The impact of each input parameter affecting the pressure drop is quantified using the RF algorithm.

The Capacity of the Road Network: Data Collection and Statistical Analysis of Traffic Characteristics

The possibilities of collecting the necessary information using multi-touch cameras and ways to improve road traffic data collection are considered. An increase in the number of vehicles leads to traffic jams, which in turn leads to an increase in travel time, additional fuel consumption and other negative consequences. To solve this problem, it is necessary to have a reliable information collection system and apply modern effective methods of processing the collected information. The technology considered in the article allows taking into account pedestrians crossing the intersection. The purpose of this article is to determine the most important traffic characteristics that affect the traffic capacity of the intersection, in other words, the actual number of passing cars. Throughput is taken as a dependent variable. Based on the results of the regression analysis, a model was developed to predict the intersection throughput taking into account the most important traffic characteristics. Besides, this model is based on the fuzzy logic method and using the Fuzzy TECH 5.81d Professional Edition computer program.

A Hyperheuristic Approach for Location-Routing Problem of Cold Chain Logistics considering Fuel Consumption

In response to violent market competition and demand for low-carbon economy, cold chain logistics companies have to pay attention to customer satisfaction and carbon emission for better development. In this paper, a biobjective mathematical model is established for cold chain logistics network in consideration of economic, social, and environmental benefits; in other words, the total cost and distribution period of cold chain logistics are optimized, while the total cost consists of cargo damage cost, refrigeration cost of refrigeration equipment, transportation cost, fuel consumption cost, penalty cost of time window, and operation cost of distribution centres. One multiobjective hyperheuristic optimization framework is proposed to address this multiobjective problem. In the framework, four selection strategies and four acceptance criteria for solution set are proposed to improve the performance of the multiobjective hyperheuristic framework. As known from a comparative study, the proposed algorithm had better overall performance than NSGA-II. Furthermore, instances of cold chain logistics are modelled and solved, and the resulting Pareto solution set offers diverse options for a decision maker to select an appropriate cold chain logistics distribution network in the interest of the logistics company.

Traversability Assessment and Trajectory Planning of Unmanned Ground Vehicles with Suspension Systems on Rough Terrain

This paper presents a traversability assessment method and a trajectory planning method. They are key features for the navigation of an unmanned ground vehicle (UGV) in a non-planar environment. In this work, a 3D light detection and ranging (LiDAR) sensor is used to obtain the geometric information about a rough terrain surface. For a given SE(2) pose of the vehicle and a specific vehicle model, the SE(3) pose of the vehicle is estimated based on LiDAR points, and then a traversability is computed. The traversability tells the vehicle the effects of its interaction with the rough terrain. Note that the traversability is computed on demand during trajectory planning, so there is not any explicit terrain discretization. The proposed trajectory planner finds an initial path through the non-holonomic A*, which is a modified form of the conventional A* planner. A path is a sequence of poses without timestamps. Then, the initial path is optimized in terms of the traversability, using the method of Lagrange multipliers. The optimization accounts for the model of the vehicle's suspension system. Therefore, the optimized trajectory is dynamically feasible, and the trajectory tracking error is small. The proposed methods were tested in both the simulation and the real-world experiments. The simulation experiments were conducted in a simulator called Gazebo, which uses a physics engine to compute the vehicle motion. The experiments were conducted in various non-planar experiments. The results indicate that the proposed methods could accurately estimate the SE(3) pose of the vehicle. Besides, the trajectory cost of the proposed planner was lower than the trajectory costs of other state-of-the-art trajectory planners.

Sensor Reliability in Cyber-Physical Systems Using Internet-of-Things Data: A Review and Case Study

Nowadays, reliability of sensors is one of the most important challenges for widespread application of Internet-of-things data in key emerging fields such as the automotive and manufacturing sectors. This paper presents a brief review of the main research and innovation actions at the European level, as well as some on-going research related to sensor reliability in cyber-physical systems (CPS). The research reported in this paper is also focused on the design of a procedure for evaluating the reliability of Internet-of-Things sensors in a cyber-physical system. The results of a case study of sensor reliability assessment in an autonomous driving scenario for the automotive sector are also shown. A co-simulation framework is designed in order to enable real-time interaction between virtual and real sensors. The case study consists of an IoT LiDAR-based collaborative map in order to assess the CPS-based co-simulation framework. Specifically, the sensor chosen is the Ibeo Lux 4-layer LiDAR sensor with IoT added capabilities. The modeling library for predicting error with machine learning methods is implemented at a local level, and a self-learning-procedure for decision-making based on Q-learning runs at a global level. The study supporting the experimental evaluation of the co-simulation framework is presented using simulated and real data. The results demonstrate the effectiveness of the proposed method for increasing sensor reliability in cyber-physical systems using Internet-of-Things data.

Socially Compliant Path Planning for Robotic Autonomous Luggage Trolley Collection at Airports

This paper describes a socially compliant path planning scheme for robotic autonomous luggage trolley collection at airports. The robot is required to efficiently collect all assigned luggage trolleys in a designated area, while avoiding obstacles and not offending the pedestrians. This path planning problem is formulated in this paper as a Traveling Salesman Problem (TSP). Different from the conventional solutions to the TSP, in which the Euclidean distance between two sites is used as the metric, a high-dimensional metric including the factor of pedestrians’ feelings is applied in this work. To obtain the new metric, a novel potential function is firstly proposed to model the relationship between the robot, luggage trolleys, obstacles, and pedestrians. The Social Force Model (SFM) is utilized so that the pedestrians can bring extra influence on the potential field, different from ordinary obstacles. Directed by the attractive and repulsive force generated from the potential field, a number of paths connecting the robot and the luggage trolley, or two luggage trolleys, can be obtained. The length of the generated path is considered as the new metric. The Self-Organizing Map (SOM) satisfies the job of finding a final path to connect all luggage trolleys and the robot located in the potential field, as it can find the intrinsic connection in the high dimensional space. Therefore, while incorporating the new metric, the SOM is used to find the optimal path in which the robot can collect the assigned luggage trolleys in sequence. As a demonstration, the proposed path planning method is implemented in simulation experiments, showing an increase of efficiency and efficacy.

Rollover-Free Path Planning for Off-Road Autonomous Driving

Perception, planning, and control are three enabling technologies to achieve autonomy in autonomous driving. In particular, planning provides vehicles with a safe and collision-free path towards their destinations, accounting for vehicle dynamics, maneuvering capabilities in the presence of obstacles, traffic rules, and road boundaries. Existing path planning algorithms can be divided into two stages: global planning and local planning. In the global planning stage, global routes and the vehicle states are determined from a digital map and the localization system. In the local planning stage, a local path can be achieved based on a global route and surrounding information obtained from sensors such as cameras and LiDARs. In this paper, we present a new local path planning method, which incorporates a vehicle’s time-to-rollover model for off-road autonomous driving on different road profiles for a given predefined global route. The proposed local path planning algorithm uses a 3D occupancy grid and generates a series of 3D path candidates in the s-p coordinate system. The optimal path is then selected considering the total cost of safety, including obstacle avoidance, vehicle rollover prevention, and comfortability in terms of path smoothness and continuity with road unevenness. The simulation results demonstrate the effectiveness of the proposed path planning method for various types of roads, indicating its wide practical applications to off-road autonomous driving.

A Novel Cooperative Path Planning for Multi-robot Persistent Coverage with Obstacles and Coverage Period Constraints

In this paper, a multi-robot persistent coverage of the region of interest is considered, where persistent coverage and cooperative coverage are addressed simultaneously. Previous works have mainly concentrated on the paths that allow for repeated coverage, but ignored the coverage period requirements of each sub-region. In contrast, this paper presents a combinatorial approach for path planning, which aims to cover mission domains with different task periods while guaranteeing both obstacle avoidance and minimizing the number of robots used. The algorithm first deploys the sensors in the region to satisfy coverage requirements with minimum cost. Then it solves the travelling salesman problem to obtain the frame of the closed path. Finally, the approach partitions the closed path into the fewest segments under the coverage period constraints, and it generates the closed route for each robot on the basis of portioned segments of the closed path. Therefore, each robot can circumnavigate one closed route to cover the different task areas completely and persistently. The numerical simulations show that the proposed approach is feasible to implement the cooperative coverage in consideration of obstacles and coverage period constraints, and the number of robots used is also minimized.

Enabling Off-Road Autonomous Navigation-Simulation of LIDAR in Dense Vegetation

Machine learning techniques have accelerated the development of autonomous navigation algorithms in recent years, especially algorithms for on-road autonomous navigation. However, off-road navigation in unstructured environments continues to challenge autonomous ground vehicles. Many off-road navigation systems rely on LIDAR to sense and classify the environment, but LIDAR sensors often fail to distinguish navigable vegetation from non-navigable solid obstacles. While other areas of autonomy have benefited from the use of simulation, there has not been a real-time LIDAR simulator that accounted for LIDAR–vegetation interaction. In this work, we outline the development of a real-time, physics-based LIDAR simulator for densely vegetated environments that can be used in the development of LIDAR processing algorithms for off-road autonomous navigation. We present a multi-step qualitative validation of the simulator, which includes the development of an improved statistical model for the range distribution of LIDAR returns in grass. As a demonstration of the simulator’s capability, we show an example of the simulator being used to evaluate autonomous navigation through vegetation. The results demonstrate the potential for using the simulation in the development and testing of algorithms for autonomous off-road navigation.

Smart Sensing of Pavement Temperature Based on Low-Cost Sensors and V2I Communications

Nowadays, the preservation, maintenance, rehabilitation, and improvement of road networks are key issues. Pavement condition is highly affected by environmental factors such as temperature and humidity, hence the importance of building databases enriched with real-time information from monitoring systems that enable the analysis and modeling of the road properties. Information and communication technologies, and specifically wireless sensor networks and computational intelligence methods, are enabling the design of new monitoring systems. The main goal of this work is the design of a pavement monitoring system for measuring temperature at internal layers. The proposed solution is based on low-cost and robust temperature sensors, vehicle-to-infrastructure communications, allowing one to transmit information directly from probes to a moving auscultation vehicle, and a neural network-based model for prediction pavement temperature. User requirements drive probes’ design to a modular device, with easy installation, low cost, and reduced energy consumption. Results of the test and validation experiments show both the benefits and viability of the proposed system, which reflect in an accuracy improvement and reduction in routine test duration. Finally, data collected over a year is applied to assess the performance of BELLS3 models and the suggested neural network for predicting pavement temperature. The dynamic behavior of the predicted temperature and the mean absolute error of the neural network-based model are better than the BELL3 model, demonstrating the suitability of the proposed pavement monitoring system.

Self-Tuning Method for Increased Obstacle Detection Reliability Based on Internet of Things LiDAR Sensor Models

On-chip LiDAR sensors for vehicle collision avoidance are a rapidly expanding area of research and development. The assessment of reliable obstacle detection using data collected by LiDAR sensors has become a key issue that the scientific community is actively exploring. The design of a self-tuning methodology and its implementation are presented in this paper, to maximize the reliability of LiDAR sensors network for obstacle detection in the ‘Internet of Things’ (IoT) mobility scenarios. The Webots Automobile 3D simulation tool for emulating sensor interaction in complex driving environments is selected in order to achieve that objective. Furthermore, a model-based framework is defined that employs a point-cloud clustering technique, and an error-based prediction model library that is composed of a multilayer perceptron neural network, and k-nearest neighbors and linear regression models. Finally, a reinforcement learning technique, specifically a Q-learning method, is implemented to determine the number of LiDAR sensors that are required to increase sensor reliability for obstacle localization tasks. In addition, a IoT driving assistance user scenario, connecting a five LiDAR sensor network is designed and implemented to validate the accuracy of the computational intelligence-based framework. The results demonstrated that the self-tuning method is an appropriate strategy to increase the reliability of the sensor network while minimizing detection thresholds.