Fish can save energy via proprioceptive sensing

Learning obstacle avoidance and predation in complex reef environments with deep reinforcement learning

The reef ecosystem plays a vital role as a habitat for fish species with limited swimming capabilities, serving not only as a sanctuary and food source but also influencing their behavioral tendencies. Understanding the intricate mechanism through which fish adeptly navigate the moving targets within reef environments within complex water flow, all while evading obstacles and maintaining stable postures, has remained a challenging and prominent subject in the realms of fish behavior, ecology, and biomimetics alike. An integrated simulation framework is used to investigate fish predation problems within intricate environments, combining deep reinforcement learning algorithms (DRL) with high-precision fluid-structure interaction numerical methods-immersed boundary lattice Boltzmann method (lB-LBM). The Soft Actor-Critic (SAC) algorithm is used to improve the intelligent fish's capacity for random exploration, tackling the multi-objective sparse reward challenge inherent in real-world scenarios. Additionally, a reward shaping method tailored to its action purposes has been developed, capable of capturing outcomes and trend characteristics effectively. The convergence and robustness advantages of the method elucidated in this paper are showcased through two case studies: one addressing fish capturing randomly moving targets in hydrostatic flow field, and the other focusing on fish counter-current foraging in reef environments to capture drifting food. A comprehensive analysis was conducted of the influence and significance of various reward types on the decision-making processes of intelligent fish within intricate environments.

Energetic benefits in coordinated circular swimming motion of two swimmers

The coordinated movement of multiple swimmers is a crucial component of fish schools. Fish swimming in different formations, such as tandem, side-by-side, diamond, and phalanx, can achieve significant energetic advantages. However, the energetic benefits of nonstraight swimming behaviors, such as the collective motion of a milling pattern, are not well understood. To fill in this gap, we consider two swimmers in circular tracks, controlled by a PID approach to reach stable configurations. Our study finds that the optimal phase is affected by circumferential effects, and that substantial energy savings can result from both propulsion and turning. We also explore the radial effect in terms of energetic benefits. In a milling pattern, the inner swimmers can easily gain a certain energetic benefit (-8%), while their peers on the outside must be close enough to the inner swimmer with a proper phase to gain the energetic benefit (-14%). When the radial spacing becomes larger or is in an unmatched phase, the swimming of the outer swimmers becomes more laborious (+16%). Our results indicate that swimmers who maintain a matched phase and minimum radial effect obtain the highest energetic benefits (-26%). These findings highlight the energetic benefits of swimmers, even in a milling pattern, where the position difference dominates the extent of benefit.

A Survey on Reinforcement Learning Methods in Bionic Underwater Robots

Bionic robots possess inherent advantages for underwater operations, and research on motion control and intelligent decision making has expanded their application scope. In recent years, the application of reinforcement learning algorithms in the field of bionic underwater robots has gained considerable attention, and continues to grow. In this paper, we present a comprehensive survey of the accomplishments of reinforcement learning algorithms in the field of bionic underwater robots. Firstly, we classify existing reinforcement learning methods and introduce control tasks and decision making tasks based on the composition of bionic underwater robots. We further discuss the advantages and challenges of reinforcement learning for bionic robots in underwater environments. Secondly, we review the establishment of existing reinforcement learning algorithms for bionic underwater robots from different task perspectives. Thirdly, we explore the existing training and deployment solutions of reinforcement learning algorithms for bionic underwater robots, focusing on the challenges posed by complex underwater environments and underactuated bionic robots. Finally, the limitations and future development directions of reinforcement learning in the field of bionic underwater robots are discussed. This survey provides a foundation for exploring reinforcement learning control and decision making methods for bionic underwater robots, and provides insights for future research.

A comprehensive review on fish-inspired robots

Recently, the increasing interest in underwater exploration motivates the development of aquatic unmanned vehicles. To execute hazardous tasks in an unknown or even hostile environment, researchers have directed on developing biomimetic robots inspired by the extraordinary maneuverability, cruising speed, and propulsion efficiency of fish. Nevertheless, the performance of current prototypes still has gaps compared with that of real fishes. In this review, recent approaches in structure designs, actuators, and sensors are presented. In addition, the theoretical methods for modeling the robotic fishes are consolidated, and the control strategies are offered. Finally, the current challenges are summarized, and possible future directions are deeply discussed. It is expected that the emergence of new engineering and biological technologies will enhance the field of robotic fish for further advancement.

Real-time parametric estimation of periodic wake-foil interactions using bioinspired pressure sensing and machine learning

Periodic wake-foil interactions occur in the collective swimming of bio-inspired robots. Wake interaction pattern estimation (and control) is crucial to thrust enhancement and propulsive efficiency optimization. In this paper, we study the wake interaction pattern estimation of two flapping foils in tandem configurations. The experiments are conducted at a Reynolds number of 1.41 × 10⁴in a water channel. A modified wake-foil phase parameter Φ, which unifies the influences of inter-foil distanceL_x, motion phase difference Δφand wake convection velocityU_v, is introduced to describe the wake interaction patterns parametrically. We use a differential pressure sensor on the downstream foil to capture wake interaction characteristics. Data sets at different tandem configurations are collected. The wake-foil phase Φ is used to label the pressure signals. A one-dimensional convolutional neural networks (1D-CNN) model is used to learn an end-to-end mapping between the raw pressure measurements and the wake-foil phase Φ. The trained 1D-CNN model shows accurate estimations (average error 3.5%) on random wake interaction patterns and is fast enough (within 40 ms). Then the trained 1D-CNN model is applied to online thrust enhancement control of a downstream foil swimming in a periodic wake. Synchronous force monitoring and flow visualization demonstrate the effectiveness of the 1D-CNN model. The limitations of the model are discussed. The proposed approach can be applied to the online estimation and control of wake interactions in the collective swimming and flying of biomimetic robots.