Multi-AUV Target Search Based on Bioinspired Neurodynamics Model in 3-D Underwater Environments

Distributed Cooperative Quantum Learning for Discrete-Time Multiagent Source Exploration With Information Prompts

In light of optimization theory and swarm evolutionary schemes, under multiple single-integrator mobile agents equipped with sensors and prompters, this article addresses a discrete-time multiagent source exploration problem with information prompts. Regarding information prompts as constraints on the unknown target, by virtue of penalty function skills (PFSs) and sequential unconstrained minimization techniques (SUMTs), the agents are driven toward the source under the guidance of the control strategy. In two cases of available and unavailable gradient information, a quantum potential well, an average optimal position estimator (AOPE), and a global optimal position estimator (GOPE) are introduced into swarm evolutionary schemes with a periodically oscillating weight, such that distributed cooperative quantum learning (DCQL) policy is proposed as a control strategy under communication restrictions, where AOPE and GOPE are developed relying on distributed consensus theory. In particular, when the gradient is unavailable, we put forth an adaptive generalized Bernstein neural network (AGBNN) to replace it based on excellent properties of Bernstein polynomials and adaptive approaches. Further, a performance analysis for the proposed policy is executed on the convergence and computational complexity, which ensures the accuracy and efficiency of the source exploration in theory. Ultimately, a simulation test is carried out, and the results validate the practicability and effectiveness of the offered method.

DRL-Searcher: A Unified Approach to Multirobot Efficient Search for a Moving Target

This article studies the multirobot efficient search (MuRES) for a nonadversarial moving target problem, whose objective is usually defined as either minimizing the target's expected capture time or maximizing the target's capture probability within a given time budget. Different from canonical MuRES algorithms, which target only one specific objective, our proposed algorithm, named distributional reinforcement learning-based searcher (DRL-Searcher), serves as a unified solution to both MuRES objectives. DRL-Searcher employs distributional reinforcement learning (DRL) to evaluate the full distribution of a given search policy's return, that is, the target's capture time, and thereafter makes improvements with respect to the particularly specified objective. We further adapt DRL-Searcher to the use case without the target's real-time location information, where only the probabilistic target belief (PTB) information is provided. Lastly, the recency reward is designed for implicit coordination among multiple robots. Comparative simulation results in a range of MuRES test environments show the superior performance of DRL-Searcher to state of the arts. Additionally, we deploy DRL-Searcher to a real multirobot system for moving target search in a self-constructed indoor environment with satisfying results.

The Unified Task Assignment for Underwater Data Collection With Multi-AUV System: A Reinforced Self-Organizing Mapping Approach

This article deals with the task assignment problem for multiple autonomous underwater vehicles to efficiently collect underwater data from sensors. We formulate a unified framework to consistently address the heterogeneous task assignment problem (nonemergency and emergency cases) without strictly distinguishing the mixed cases. First, a unified problem, which bridges the gap between different constraints and optimization objectives of different cases, is constructed. Then, the proposed reinforced self-organizing mapping algorithm is reinforced in three aspects: the regional learning rate, the self-configuring neuron (SCN) strategy, and the workload balance mechanism. Specifically, the proposed regional learning rate comprehensively considers the individual worth of tasks and the topology to generate the regional learning rate of dynamic task regions, which consists of dynamic remaining tasks and the reconstructed topology. Based on this idea, the constructed unified problem can be solved consistently. Furthermore, the proposed SCN strategy optimizes the neuron population both in quality and quantity, and guides the update of neurons with enriched historical information to improve the mapping ability. This strategy greatly improves learning efficiency and applicability in a wide range of scenarios. Meanwhile, the proposed workload balance mechanism takes into consideration of both the work capability and consumed energy to extend the continuous working capability. The numerical results validate the effectiveness and adaptability of the proposed unified task assignment framework.

Research on Obstacle Detection and Avoidance of Autonomous Underwater Vehicle Based on Forward-Looking Sonar

Due to the complexity of the ocean environment, an autonomous underwater vehicle (AUV) is disturbed by obstacles when performing tasks. Therefore, the research on underwater obstacle detection and avoidance is particularly important. Based on the images collected by a forward-looking sonar on an AUV, this article proposes an obstacle detection and avoidance algorithm. First, a deep learning-based obstacle candidate area detection algorithm is developed. This algorithm uses the You Only Look Once (YOLO) v3 network to determine obstacle candidate areas in a sonar image. Then, in the determined obstacle candidate areas, the obstacle detection algorithm based on the improved threshold segmentation algorithm is used to detect obstacles accurately. Finally, using the obstacle detection results obtained from the sonar images, an obstacle avoidance algorithm based on deep reinforcement learning (DRL) is developed to plan a reasonable obstacle avoidance path of an AUV. Experimental results show that the proposed algorithms improve obstacle detection accuracy and processing speed of sonar images. At the same time, the proposed algorithms ensure AUV navigation safety in a complex obstacle environment.

Formation Control and Tracking of Mobile Robots using Distributed Estimators and A Biologically Inspired Approach

This paper investigates the formation control problem for multiple nonholonomic wheeled mobile robots using distributed estimators and a biologically inspired approach. The formation pattern of the system adopts leader-follower structure and the communication topology among the multi-robot system is modelled by an undirected graph. In our proposed methodology, first, we develop an adaptive trajectory tracking control for the leader robot to follow the desired trajectory. Second, a distributed estimator is designed for each follower mobile robot, which uses its own information to estimate the leader's states, such as position, orientation, and linear velocity. Then, distributed formation tracking control laws are designed based on the distributed estimator. Furthermore, a bioinspired controller is developed to address the impractical velocity jump problem. The closed-loop system stability is analysed with the Lyapunov stability theory showing that tracking errors are asymptotically converge to zero. Finally, simulation results are provided to demonstrate the effectiveness of the proposed methods.

Hierarchical and Stable Multiagent Reinforcement Learning for Cooperative Navigation Control

We solve an important and challenging cooperative navigation control problem, Multiagent Navigation to Unassigned Multiple targets (MNUM) in unknown environments with minimal time and without collision. Conventional methods are based on multiagent path planning that requires building an environment map and expensive real-time path planning computations. In this article, we formulate MNUM as a stochastic game and devise a novel multiagent deep reinforcement learning (MADRL) algorithm to learn an end-to-end solution, which directly maps raw sensor data to control signals. Once learned, the policy can be deployed onto each agent, and thereby, the expensive online planning computations can be offloaded. However, to solve MNUM, traditional MADRL suffers from large policy solution space and nonstationary environment when agents make decisions independently and concurrently. Accordingly, we propose a hierarchical and stable MADRL algorithm. The hierarchical learning part introduces a two-layer policy model to reduce the solution space and uses an interlaced learning paradigm to learn two coupled policies. In the stable learning part, we propose to learn an extended action-value function that implicitly incorporates estimations of other agents' actions, based on which the environment's nonstationarity caused by other agents' changing policies can be alleviated. Extensive experiments demonstrate that our method can converge in a fast way and generate more efficient cooperative navigation policies than comparable methods.

A Multi-AUV Maritime Target Search Method for Moving and Invisible Objects Based on Multi-Agent Deep Reinforcement Learning

Target search for moving and invisible objects has always been considered a challenge, as the floating objects drift with the flows. This study focuses on target search by multiple autonomous underwater vehicles (AUV) and investigates a multi-agent target search method (MATSMI) for moving and invisible objects. In the MATSMI algorithm, based on the multi-agent deep deterministic policy gradient (MADDPG) method, we add spatial and temporal information to the reinforcement learning state and set up specialized rewards in conjunction with a maritime target search scenario. Additionally, we construct a simulation environment to simulate a multi-AUV search for the floating object. The simulation results show that the MATSMI method has about 20% higher search success rate and about 70 steps shorter search time than the traditional search method. In addition, the MATSMI method converges faster than the MADDPG method. This paper provides a novel and effective method for solving the maritime target search problem.

A Novel Energy-Saving Route Planning Algorithm for Marine Vehicles

Recently, the ship energy consumption issue has been in the spotlight. Meanwhile, new conventions and standards of ship energy conservation as well as ship emission reduction have been continuously introduced by the International Maritime Organization and governments around the world. Energy conservation is a crucial factor in ship route planning. This paper proposes an improved ant colony algorithm that can handle a complicated ocean current environment and find an optimal energy-saving path. Firstly, this paper focuses on both the path length and energy consumption. To reduce energy consumption, a comprehensive energy cost function is introduced by a conventional ant colony algorithm. In addition, this paper analyzes the influence of the ocean current speed and setting on a marine vehicle and establishes an improved ant colony algorithm model based on the influence of the ocean current. Finally, simulation tests are carried out in Lianyungang and its surrounding areas to test the energy consumption performance of the improved ant colony algorithm. The simulation result shows that the improved ant colony algorithm can significantly reduce the energy consumption by 3.326% compared with the traditional algorithm.

Octopus-Inspired Microgripper for Deformation-Controlled Biological Sample Manipulation

Predators in nature grip their prey in different ways, which give innovational ideas of gripping approaches in industrial applications. Octopus performs flexible gripping with the help of vacuum grippers, suction cups, which inspired a new type of microgripper for biological sample micromanipulation. The proposed gripper consists of a glass pipette and a pump driven by a step-motor. The step-motor is controlled with adaptive robust control to adjust the gripping pressure applied on the biological sample. A dynamic model is developed for the biological sample aiming for better deformation control performance. A visual detection algorithm is developed for data processing to identify the parameters in the dynamic model and the detection result of visual algorithm is also used as feedback of adaptive robust control, which diminishes the negative influence of parameter and model uncertainties. Zebrafish larva was used as the testing sample for experiment and the corresponding parameters were identified experimentally. The experimental results correlated well with the model predicted deformation curve and visual detection algorithm provided promising accuracy, which is less than [Formula: see text]. Adaptive robust control provides fast and accuracy response in point-to-point deformation testing, and the average responding time is less than 30 s and the average error is no larger than 1 pixel.

Attitude Stabilization Control of Autonomous Underwater Vehicle Based on Decoupling Algorithm and PSO-ADRC

Autonomous Underwater Vehicle are widely used in industries, such as marine resource exploitation and fish farming, but they are often subject to a large amount of interference which cause poor control stability, while performing their tasks. A decoupling control algorithm is proposed and A single control volume–single attitude angle model is constructed for the problem of severe coupling in the control system of attitude of six degrees of freedom Autonomous Underwater Vehicle. Aiming at the problem of complex Active Disturbance Rejection Control (ADRC) adjustment relying on manual experience, the PSO-ADRC algorithm is proposed to realize the automatic adjustment of its parameters, which improves the anti-interference ability and control accuracy of Autonomous Underwater Vehicle in dynamic environment. The anti-interference ability and control accuracy of the method were verified through experiments.

Gradient Descent-Based Adaptive Learning Control for Autonomous Underwater Vehicles With Unknown Uncertainties

This article investigates the adaptive learning control problem for a class of nonlinear autonomous underwater vehicles (AUVs) with unknown uncertainties. The unknown nonlinear functions in the AUVs are approximated by radial basis function neural networks (RBFNNs), in which the weight updating laws are designed via gradient descent algorithm. The proposed gradient descent-based control scheme guarantees the semiglobal uniform ultimate boundedness (SUUB) of the system and the fast convergence of the weight updating laws. In order to reduce the computational burden during the backstepping control design process, the command-filter-based design technique is incorporated into the adaptive learning control strategy. Finally, simulation studies are given to demonstrate the effectiveness of the proposed method.

A Self-Learning Immune Co-Evolutionary Network for Multiple Escaping Targets Search With Random Observable Conditions

The search for multiple escaping targets is a significant issue of cooperative control in multi-agent systems since targets consciously seek to avoid being captured. Moreover, the assumption of continuous observations in existing works is not always suitable due to the limit of measuring equipment and uncertain movement of targets. Therefore, the problem with searching for escaping targets, which can be more aptly labeled "multiple escaping-targets search with random observation conditions" (MESROC), is difficult to address by conventional methods. Inspired by machine learning and the immune response mechanism of human bodies, a self-learning immune co-evolutionary network (SLICEN) is proposed. The SLICEN consists mainly of an immune cellular network (ICN) and an immune learning algorithm (ILA). The ICN provides feasible solutions to MESROC. Different kinds of network models are introduced to work as an ICN, such as convolutional neural networks, extreme learning machines, and support vector machines. The ILA evaluates the performance of feasible solutions and selects the optimal ones to further strengthen ICN reversely. Solutions are repeatedly improved through the co-evolution of ICN and ILA. An essential distinction to conventional machine learning approaches is that SLICEN works well without training samples. Simulations and comparisons demonstrate that patterns of advanced cooperative behavior among searchers function properly. SLICEN is an efficient method for solving MESROC.

A Novel Double Layered Hybrid Multi-Robot Framework for Guidance and Navigation of Unmanned Surface Vehicles in a Practical Maritime Environment

Formation control and cooperative motion planning are two major research areas currently being used in multi robot motion planning and coordination. The current study proposes a hybrid framework for guidance and navigation of swarm of unmanned surface vehicles (USVs) by combining the key characteristics of formation control and cooperative motion planning. In this framework, two layers of offline planning and online planning are integrated and applied on a practical marine environment. In offline planning, an optimal path is generated from a constrained A* path planning approach, which is later smoothed using a spline. This optimal trajectory is fed as an input for the online planning where virtual target (VT) based multi-agent guidance framework is used to navigate the swarm of USVs. This VT approach combined with a potential theory based swarm aggregation technique provides a robust methodology of global and local collision avoidance based on known positions of the USVs. The combined approach is evaluated with the different number of USVs to understand the effectiveness of the approach from the perspective of practicality, safety and robustness.

An Energy-Efficient Redundant Transmission Control Clustering Approach for Underwater Acoustic Networks

Underwater Acoustic Network (UAN) is an emerging technology with attractive applications. In such type of networks, the control-overhead, redundant inner-network transmissions management, and data-similarity are still very challenging. The cluster-based frameworks manage the control-overhead and redundant inner-network transmissions persuasively. However, the current clustering protocols consume a big part of their energy resources in data-similarity as these protocols periodically sense and forward the same information. In this paper, we introduce a novel two-level Redundant Transmission Control (RTC) approach that ensures the data-similarity using some statistical tests with an appropriate degree of confidence. Later, the Cluster Head (CH) and the Region Head (RH) remove the data-similarity from the original data before forwarding it to the next level. We also introduce a new spatiotemporal and dynamic CH role rotation technique which is capable to adjust the drifted field nodes because of water current movements. The beauty of the proposed model is that the RH controls the communications and redundant transmission between the CH and Mobile Sink (MS), while the CH controls the redundant inner-network transmissions and data-similarity between the cluster members. We conduct simulations to evaluate the performance of our designed framework under different criteria such as average end-to-end delay, the packet delivery ratio, and energy consumption of the network with respect to the recent schemes. The presented results reveal that the proposed model outperforms the current approaches in terms of the selected metrics.

A Hybrid Energy Equating Game for Energy Management in the Internet of Underwater Things

The Internet of Underwater Things (IoUT) is an evolving class of Internet of Things and it is considered the basic unit for the development of smart cities. To support the idea of IoUT, an Underwater Sensor Network (USN) has emerged as a potential technology that has attractive and updated applications for underwater environment monitoring. In such networks, route selection and cluster-head management are still challenging. As the optimal routes always lead to congestion and longer delays while the cluster-head mismanagement leads to ending the USN lifespan earlier. In this paper, we propose a cooperative clustering game that is based upon energy heterogeneity and a penalty mechanism to deal with the cluster head mismanagement issue. Then, we use a non-cooperative evolutionary game for the best relay selection; the results prove that this utility function is the most suitable solution for the relay selection and its strategy selection converges to Nash Equilibrium. The proposed framework is compared with recent schemes using different quality measures and we found that our proposed framework performs favorably against the existing schemes for all of the evaluation metrics.

An Improved DSA-Based Approach for Multi-AUV Cooperative Search

Multi-AUV cooperative target search problem in unknown 3D underwater environment is not only a research hot spot but also a challenging task. To complete this task, each autonomous underwater vehicle (AUV) needs to move quickly without collision and cooperate with other AUVs to find the target. In this paper, an improved dolphin swarm algorithm- (DSA-) based approach is proposed, and the search problem is divided into three stages, namely, random cruise, dynamic alliance, and team search. In the proposed approach, the Levy flight method is used to provide a random walk for AUV to detect the target information in the random cruise stage. Then the self-organizing map (SOM) neural network is used to build dynamic alliances in real time. Finally, an improved DSA algorithm is presented to realize the team search. Furthermore, some simulations are conducted, and the results show that the proposed approach is capable of guiding multi-AUVs to achieve the target search task in unknown 3D underwater environment efficiently.

Visual detection and feature recognition of underwater target using a novel model-based method

Imaging is an important means to explore the ocean for underwater robotics. The diffuse attenuation coefficient of light in water is one of the most important optical properties of seawater. This article presents a model-based method to analyze the causes of distortion of underwater images. We built a platform for underwater image acquisition and target recognition. The model coefficients were calibrated with images captured underwater and in air. Experiments were carried out to verify the designed algorithm and the transmission error model. The experiments show that the presented method works well in improving the accuracy of feature extraction and recognition of underwater targets.

Prevalence of and factors related to anxiety and depression symptoms among married patients with gynecological malignancies in China

OBJECTIVE

This study aims to investigate the prevalence of anxiety and depression among married patients with gynecological malignancies in China and then explores factors related to anxiety and depression.

METHODS

A cross-sectional study was conducted from July 2016 to July 2017 in Jilin Province. A total of 394 married patients with gynecological malignancies completed the Hospital Anxiety and Depression Scale (HADS). Univariate analysis and multivariate logistic regression analysis were used to evaluate the association between influencing factors of anxiety and depression. Back propagation neural networks (BP neural networks) were used to examine the ranking of these influencing factors.

RESULTS

The prevalence of anxiety and depression was 79.95% and 94.16% in married patients with gynecological malignancies. Multivariate logistic analysis revealed that patients' anxiety showed significant relationships with low income, frequency of chemotherapy, diagnostic results and low sleep quality(P < 0.05), and patients' depression showed significant relationships with low quality sleep and frequency of chemotherapy (P < 0.05). Using BP neural networks, the order of importance of these factors in influencing anxiety, from high to low, was determined to be sleep quality, frequency of chemotherapy, monthly income, diagnostic results, age and fertility demand.

CONCLUSIONS

It is important to reduce anxiety and depression in these patients by improving sleep quality and clinical treatment.

A Sampling-Based Bayesian Approach for Cooperative Multiagent Online Search With Resource Constraints

This paper presents a cooperative multiagent search algorithm to solve the problem of searching for a target on a 2-D plane under multiple constraints. A Bayesian framework is used to update the local probability density functions (PDFs) of the target when the agents obtain observation information. To obtain the global PDF used for decision making, a sampling-based logarithmic opinion pool algorithm is proposed to fuse the local PDFs, and a particle sampling approach is used to represent the continuous PDF. Then the Gaussian mixture model (GMM) is applied to reconstitute the global PDF from the particles, and a weighted expectation maximization algorithm is presented to estimate the parameters of the GMM. Furthermore, we propose an optimization objective which aims to guide agents to find the target with less resource consumptions, and to keep the resource consumption of each agent balanced simultaneously. To this end, a utility function-based optimization problem is put forward, and it is solved by a gradient-based approach. Several contrastive simulations demonstrate that compared with other existing approaches, the proposed one uses less overall resources and shows a better performance of balancing the resource consumption.

Polar Cooperative Navigation Algorithm for Multi-Unmanned Underwater Vehicles Considering Communication Delays

To solve the navigation accuracy problems of multi-Unmanned Underwater Vehicles (multi-UUVs) in the polar region, a polar cooperative navigation algorithm for multi-UUVs considering communication delays is proposed in this paper. UUVs are important pieces of equipment in ocean engineering for marine development. For UUVs to complete missions, precise navigation is necessary. It is difficult for UUVs to establish true headings because of the rapid convergence of Earth meridians and the severe polar environment. Based on the polar grid navigation algorithm, UUV navigation in the polar region can be accomplished with the Strapdown Inertial Navigation System (SINS) in the grid frame. To save costs, a leader-follower type of system is introduced in this paper. The leader UUV helps the follower UUVs to achieve high navigation accuracy. Follower UUVs correct their own states based on the information sent by the leader UUV and the relative position measured by ultra-short baseline (USBL) acoustic positioning. The underwater acoustic communication delay is quantized by the model. In this paper, considering underwater acoustic communication delay, the conventional adaptive Kalman filter (AKF) is modified to adapt to polar cooperative navigation. The results demonstrate that the polar cooperative navigation algorithm for multi-UUVs that considers communication delays can effectively navigate the sailing of multi-UUVs in the polar region.

Task Assignment and Path Planning for Multiple Autonomous Underwater Vehicles Using 3D Dubins Curves †

This paper investigates the task assignment and path planning problem for multiple AUVs in three dimensional (3D) underwater wireless sensor networks where nonholonomic motion constraints of underwater AUVs in 3D space are considered. The multi-target task assignment and path planning problem is modeled by the Multiple Traveling Sales Person (MTSP) problem and the Genetic Algorithm (GA) is used to solve the MTSP problem with Euclidean distance as the cost function and the Tour Hop Balance (THB) or Tour Length Balance (TLB) constraints as the stop criterion. The resulting tour sequences are mapped to 2D Dubins curves in the X−Y plane, and then interpolated linearly to obtain the Z coordinates. We demonstrate that the linear interpolation fails to achieve G1 continuity in the 3D Dubins path for multiple targets. Therefore, the interpolated 3D Dubins curves are checked against the AUV dynamics constraint and the ones satisfying the constraint are accepted to finalize the 3D Dubins curve selection. Simulation results demonstrate that the integration of the 3D Dubins curve with the MTSP model is successful and effective for solving the 3D target assignment and path planning problem.

Automatic Detection of Driver Fatigue Using Driving Operation Information for Transportation Safety

Fatigued driving is a major cause of road accidents. For this reason, the method in this paper is based on the steering wheel angles (SWA) and yaw angles (YA) information under real driving conditions to detect drivers’ fatigue levels. It analyzes the operation features of SWA and YA under different fatigue statuses, then calculates the approximate entropy (ApEn) features of a short sliding window on time series. Using the nonlinear feature construction theory of dynamic time series, with the fatigue features as input, designs a “2-6-6-3” multi-level back propagation (BP) Neural Networks classifier to realize the fatigue detection. An approximately 15-h experiment is carried out on a real road, and the data retrieved are segmented and labeled with three fatigue levels after expert evaluation, namely “awake”, “drowsy” and “very drowsy”. The average accuracy of 88.02% in fatigue identification was achieved in the experiment, endorsing the value of the proposed method for engineering applications.