Physics-informed reinforcement learning for motion control of a fish-like swimming robot

Motion control of fish-like swimming robots presents many challenges due to the unstructured environment and unmodelled governing physics of the fluid-robot interaction. Commonly used low-fidelity control models using simplified formulas for drag and lift forces do not capture key physics that can play an important role in the dynamics of small-sized robots with limited actuation. Deep Reinforcement Learning (DRL) holds considerable promise for motion control of robots with complex dynamics. Reinforcement learning methods require large amounts of training data exploring a large subset of the relevant state space, which can be expensive, time consuming, or unsafe to obtain. Data from simulations can be used in the initial stages of DRL, but in the case of swimming robots, the complexity of fluid-body interactions makes large numbers of simulations infeasible from the perspective of time and computational resources. Surrogate models that capture the primary physics of the system can be a useful starting point for training a DRL agent which is subsequently transferred to train with a higher fidelity simulation. We demonstrate the utility of such physics-informed reinforcement learning to train a policy that can enable velocity and path tracking for a planar swimming (fish-like) rigid Joukowski hydrofoil. This is done through a curriculum where the DRL agent is first trained to track limit cycles in a velocity space for a representative nonholonomic system, and then transferred to train on a small simulation data set of the swimmer. The results show the utility of physics-informed reinforcement learning for the control of fish-like swimming robots.

A Review of Robotic Fish Based on Smart Materials

The present study focuses on summarizing the recent advancements in the field of fish swimming mode research and bionic robotic fish prototypes based on smart materials. It has been widely acknowledged that fish exhibit exceptional swimming efficiency and manoeuvrability compared to conventional underwater vehicles. In the pursuit of developing autonomous underwater vehicles (AUVs), conventional experimental methods often prove to be complex and expensive. Hence, the utilization of computer simulations for hydrodynamic modelling provides a cost-effective and efficient approach for analysing the swimming behaviour of bionic robotic fish. Additionally, computer simulations can provide data that are difficult to obtain through experimental methods. Smart materials, which integrate perception, drive, and control functions, are increasingly being applied to bionic robotic fish research. However, the utilization of smart materials in this field is still an area of ongoing research and several challenges remain unresolved. This study provides an overview of the current state of research on fish swimming modes and the development of hydrodynamic modelling. The application of four distinct types of smart materials in bionic robotic fish is then reviewed, with a focus on analysing the advantages and disadvantages of each material in driving swimming behaviour. In conclusion, the paper highlights the key technical challenges that must be addressed for the practical implementation of bionic robotic fish and provides insights into the potential future directions of this field.

Emergence of robust self-organized undulatory swimming based on local hydrodynamic force sensing

Undulatory swimming represents an ideal behavior to investigate locomotion control and the role of the underlying central and peripheral components in the spinal cord. Many vertebrate swimmers have central pattern generators and local pressure-sensitive receptors that provide information about the surrounding fluid. However, it remains difficult to study experimentally how these sensors influence motor commands in these animals. Here, using a specifically designed robot that captures the essential components of the animal neuromechanical system and using simulations, we tested the hypothesis that sensed hydrodynamic pressure forces can entrain body actuation through local feedback loops. We found evidence that this peripheral mechanism leads to self-organized undulatory swimming by providing intersegmental coordination and body oscillations. Swimming can be redundantly induced by central mechanisms, and we show that, therefore, a combination of both central and peripheral mechanisms offers a higher robustness against neural disruptions than any of them alone, which potentially explains how some vertebrates retain locomotor capabilities after spinal cord lesions. These results broaden our understanding of animal locomotion and expand our knowledge for the design of robust and modular robots that physically interact with the environment.

Structural Dynamics of a Pulsed-Jet Propulsion System for Underwater Soft Robots

This paper entails the study of the pulsed-jet propulsion inspired by cephalopods in the frame of underwater bioinspired robotics. This propulsion routine involves a sequence of consecutive cycles of inflation and collapse of an elastic bladder, which, in the robotics artefact developed by the authors, is enabled by a cable-driven actuation of a deformable shell composed of rubber-like materials. In the present work an all-comprehensive formulation is derived by resorting to a coupled approach that comprises of a model of the structural dynamics of the cephalopod-like elastic bladder and a model of the pulsed-jet thrust production. The bladder, or mantle, is modelled by means of geometrically exact, axisymmetric, nonlinear shell theory, which yields an accurate estimation of the forces involved in driving the deformation of the structure in water. By coupling these results with those from a standard thrust model, the behaviour of the vehicle propelling itself in water is derived. The constitutive laws of the shell are also exploited as control laws with the scope of replicating the muscle activation routine observed in cephalopods. The model is employed to test various shapes, material properties and actuation routines of the mantle. The results are compared in terms of speed performance in order to identify suitable design guidelines. Altogether, the model is tested in more than 50 configurations, eventually providing useful insight for the development of more advanced vehicles and bringing evidence of its reliability in studying the dynamics of both man-made cephalopod-inspired robots and live specimens.

Improved Lighthill fish swimming model for bio-inspired robots: Modeling, computational aspects and experimental comparisons

The best known analytical model of swimming was originally developed by Lighthill and is known as the large amplitude elongated body theory (LAEBT). Recently, this theory has been improved and adapted to robotics through a series of studies ranging from hydrodynamic modeling to mobile multibody system dynamics. This article marks a further step towards the Lighthill theory. The LAEBT is applied to one of the best bio-inspired swimming robots yet built: the AmphiBot III, a modular anguilliform swimming robot. To that end, we apply a Newton–Euler modeling approach and focus our attention on the model of hydrodynamic forces. This model is numerically integrated in real time by using an extension of the Newton–Euler recursive forward dynamics algorithm for manipulators to a robot without a fixed base. Simulations and experiments are compared on undulatory gaits and turning maneuvers for a wide range of parameters. The discrepancies between modeling and reality do not exceed 16% for the swimming speed, while requiring only the one-time calibration of a few hydrodynamic parameters. Since the model can be numerically integrated in real time, it has significantly superior accuracy compared with computational speed ratio, and is, to the best of our knowledge, one of the most accurate models that can be used in real-time. It should provide an interesting tool for the design and control of swimming robots. The approach is presented in a self contained manner, with the concern to help the reader not familiar with fluid dynamics to get insight both into the physics of swimming and the mathematical tools that can help its modeling.

Optimally efficient swimming in hyper-redundant mechanisms: control, design, and energy recovery

Hyper-redundant mechanisms (HRMs), also known as snake-like robots, are highly adaptable during locomotion on land. Researchers are currently working to extend their capabilities to aquatic environments through biomimetic undulatory propulsion. In addition to increasing the versatility of the system, truly biomimetic swimming could also provide excellent locomotion efficiency. Unfortunately, the complexity of the system precludes the development of a functional solution to achieve this. To explore this problem, a rapid optimization process is used to generate efficient HRM swimming gaits. The low computational cost of the approach allows for multiple optimizations over a broad range of system conditions. By observing how these conditions affect optimal kinematics, a number of new insights are developed regarding undulatory swimming in robotic systems. Two key conditions are varied within the study, swimming speed and energy recovery. It is found that the swimmer mimics the speed control behaviour of natural fish and that energy recovery drastically increases the system's efficiency. Remarkably, this efficiency increase is accompanied by a distinct change in swimming kinematics. With energy recovery, the swimmer converges to a clearly anguilliform gait, without, it tends towards the carangiform mode.

The effect of non-uniform damping on flutter in axial flow and energy-harvesting strategies

The problem of energy harvesting from flutter instabilities in flexible slender structures in axial flows is considered. In a recent study, we used a reduced-order theoretical model of such a system to demonstrate the feasibility for harvesting energy from these structures. Following this preliminary study, we now consider a continuous fluid-structure system. Energy harvesting is modelled as strain-based damping, and the slender structure under investigation lies in a moderate fluid loading range, for which the flexible structure may be destabilized by damping. The key goal of this work is to analyse the effect of damping distribution and intensity on the amount of energy harvested by the system. The numerical results indeed suggest that non-uniform damping distributions may significantly improve the power-harvesting capacity of the system. For low-damping levels, clustered dampers at the position of peak curvature are shown to be optimal. Conversely for higher damping, harvesters distributed over the whole structure are more effective.