Model predictive control of non-interacting active Brownian particles

Active matter systems are strongly driven to assume non-equilibrium distributions owing to their self-propulsion, e.g., flocking and clustering. Controlling the active matter systems' spatiotemporal distributions offers exciting applications such as directed assembly, programmable materials, and microfluidic actuation. However, these applications involve environments with coupled dynamics and complex tasks, making intuitive control strategies insufficient. This necessitates the development of an automatic feedback control framework, where an algorithm determines appropriate actions based on the system's current state. In this work, we control the distribution of active Brownian particles by applying model predictive control (MPC), a model-based control algorithm that predicts future states and optimizes the control inputs to drive the system along a user-defined objective. The MPC model is based on the Smoluchowski equation with a self-propulsive convective term and an actuated spatiotemporal-varying external field that aligns particles with the applied direction, similar to a magnetic field. We apply the MPC framework to control a Brownian dynamics simulation of non-interacting active particles and illustrate the controller capabilities with two objectives: splitting and juggling sub-populations, and polar order flocking control.

Field-Directed Motion, Cargo Capture, and Closed-Loop Controlled Navigation of Microellipsoids

Microrobots have the potential for diverse applications, including targeted drug delivery and minimally invasive surgery. Despite advancements in microrobot design and actuation strategies, achieving precise control over their motion remains challenging due to the dominance of viscous drag, system disturbances, physicochemical heterogeneities, and stochastic Brownian forces. Here, a precise control over the interfacial motion of model microellipsoids is demonstrated using time-varying rotating magnetic fields. The impacts of microellipsoid aspect ratio, field characteristics, and magnetic properties of the medium and the particle on the motion are investigated. The role of mobile micro-vortices generated is highlighted by rotating microellipsoids in capturing, transporting, and releasing cargo objects. Furthermore, an approach is presented for controlled navigation through mazes based on real-time particle and obstacle sensing, path planning, and magnetic field actuation without human intervention. The study introduces a mechanism of directing motion of microparticles using rotating magnetic fields, and a control scheme for precise navigation and delivery of micron-sized cargo using simple microellipsoids as microbots.

AI-enhanced biomedical micro/nanorobots in microfluidics

Human beings encompass sophisticated microcirculation and microenvironments, incorporating a broad spectrum of microfluidic systems that adopt fundamental roles in orchestrating physiological mechanisms. In vitro recapitulation of human microenvironments based on lab-on-a-chip technology represents a critical paradigm to better understand the intricate mechanisms. Moreover, the advent of micro/nanorobotics provides brand new perspectives and dynamic tools for elucidating the complex process in microfluidics. Currently, artificial intelligence (AI) has endowed micro/nanorobots (MNRs) with unprecedented benefits, such as material synthesis, optimal design, fabrication, and swarm behavior. Using advanced AI algorithms, the motion control, environment perception, and swarm intelligence of MNRs in microfluidics are significantly enhanced. This emerging interdisciplinary research trend holds great potential to propel biomedical research to the forefront and make valuable contributions to human health. Herein, we initially introduce the AI algorithms integral to the development of MNRs. We briefly revisit the components, designs, and fabrication techniques adopted by robots in microfluidics with an emphasis on the application of AI. Then, we review the latest research pertinent to AI-enhanced MNRs, focusing on their motion control, sensing abilities, and intricate collective behavior in microfluidics. Furthermore, we spotlight biomedical domains that are already witnessing or will undergo game-changing evolution based on AI-enhanced MNRs. Finally, we identify the current challenges that hinder the practical use of the pioneering interdisciplinary technology.

Intelligent Navigation of a Magnetic Microrobot with Model-Free Deep Reinforcement Learning in a Real-World Environment

Microrobotics has opened new horizons for various applications, especially in medicine. However, it also witnessed challenges in achieving maximum optimal performance. One key challenge is the intelligent, autonomous, and precise navigation control of microrobots in fluid environments. The intelligence and autonomy in microrobot control, without the need for prior knowledge of the entire system, can offer significant opportunities in scenarios where their models are unavailable. In this study, two control systems based on model-free deep reinforcement learning were implemented to control the movement of a disk-shaped magnetic microrobot in a real-world environment. The training and results of an off-policy SAC algorithm and an on-policy TRPO algorithm revealed that the microrobot successfully learned the optimal path to reach random target positions. During training, the TRPO exhibited a higher sample efficiency and greater stability. The TRPO and SAC showed 100% and 97.5% success rates in reaching the targets in the evaluation phase, respectively. These findings offer basic insights into achieving intelligent and autonomous navigation control for microrobots to advance their capabilities for various applications.

Optimal navigation of a smart active particle: directional and distance sensing

We employ Q learning, a variant of reinforcement learning, so that an active particle learns by itself to navigate on the fastest path toward a target while experiencing external forces and flow fields. As state variables, we use the distance and direction toward the target, and as action variables the active particle can choose a new orientation along which it moves with constant velocity. We explicitly investigate optimal navigation in a potential barrier/well and a uniform/ Poiseuille/swirling flow field. We show that Q learning is able to identify the fastest path and discuss the results. We also demonstrate that Q learning and applying the learned policy works when the particle orientation experiences thermal noise. However, the successful outcome strongly depends on the specific problem and the strength of noise.

Learning to cooperate for low-Reynolds-number swimming: a model problem for gait coordination

Biological microswimmers can coordinate their motions to exploit their fluid environment-and each other-to achieve global advantages in their locomotory performance. These cooperative locomotion require delicate adjustments of both individual swimming gaits and spatial arrangements of the swimmers. Here we probe the emergence of such cooperative behaviors among artificial microswimmers endowed with artificial intelligence. We present the first use of a deep reinforcement learning approach to empower the cooperative locomotion of a pair of reconfigurable microswimmers. The AI-advised cooperative policy comprises two stages: an approach stage where the swimmers get in close proximity to fully exploit hydrodynamic interactions, followed a synchronization stage where the swimmers synchronize their locomotory gaits to maximize their overall net propulsion. The synchronized motions allow the swimmer pair to move together coherently with an enhanced locomotion performance unattainable by a single swimmer alone. Our work constitutes a first step toward uncovering intriguing cooperative behaviors of smart artificial microswimmers, demonstrating the vast potential of reinforcement learning towards intelligent autonomous manipulations of multiple microswimmers for their future biomedical and environmental applications.

Optoelectronic Trajectory Reconfiguration and Directed Self-Assembly of Self-Propelling Electrically Powered Active Particles

Self-propelling active particles are an exciting and interdisciplinary emerging area of research with projected biomedical and environmental applications. Due to their autonomous motion, control over these active particles that are free to travel along individual trajectories, is challenging. This work uses optically patterned electrodes on a photoconductive substrate using a digital micromirror device (DMD) to dynamically control the region of movement of self-propelling particles (i.e., metallo-dielectric Janus particles (JPs)). This extends previous studies where only a passive micromotor is optoelectronically manipulated with a translocating optical pattern that illuminates the particle. In contrast, the current system uses the optically patterned electrode merely to define the region within which the JPs moved autonomously. Interestingly, the JPs avoid crossing the optical region's edge, which enables constraint of the area of motion and to dynamically shape the JP trajectory. Using the DMD system to simultaneously manipulate several JPs enables to self-assemble the JPs into stable active structures (JPs ring) with precise control over the number of participating JPs and passive particles. Since the optoelectronic system is amenable to closed-loop operation using real-time image analysis, it enables exploitation of these active particles as active microrobots that can be operated in a programmable and parallelized manner.

Advanced medical micro-robotics for early diagnosis and therapeutic interventions

Recent technological advances in micro-robotics have demonstrated their immense potential for biomedical applications. Emerging micro-robots have versatile sensing systems, flexible locomotion and dexterous manipulation capabilities that can significantly contribute to the healthcare system. Despite the appreciated and tangible benefits of medical micro-robotics, many challenges still remain. Here, we review the major challenges, current trends and significant achievements for developing versatile and intelligent micro-robotics with a focus on applications in early diagnosis and therapeutic interventions. We also consider some recent emerging micro-robotic technologies that employ synthetic biology to support a new generation of living micro-robots. We expect to inspire future development of micro-robots toward clinical translation by identifying the roadblocks that need to be overcome.

Could artificial intelligence revolutionize the development of nanovectors for gene therapy and mRNA vaccines?

Gene therapy enables the introduction of nucleic acids like DNA and RNA into host cells, and is expected to revolutionize the treatment of a wide range of diseases. This growth has been further accelerated by the discovery of CRISPR/Cas technology, which allows accurate genomic editing in a broad range of cells and organisms in vitro and in vivo. Despite many advances in gene delivery and the development of various viral and non-viral gene delivery vectors, the lack of highly efficient non-viral systems with low cellular toxicity remains a challenge. The application of cutting-edge technologies such as artificial intelligence (AI) has great potential to find new paradigms to solve this issue. Herein, we review AI and its major subfields including machine learning (ML), neural networks (NNs), expert systems, deep learning (DL), computer vision and robotics. We discuss the potential of AI-based models and algorithms in the design of targeted gene delivery vehicles capable of crossing extracellular and intracellular barriers by viral mimicry strategies. We finally discuss the role of AI in improving the function of CRISPR/Cas systems, developing novel nanobots, and mRNA vaccine carriers.

Formation Techniques Used in Shape-Forming Microrobotic Systems with Multiple Microrobots: A Review

Multiple robots are used in robotic applications to achieve tasks that are impossible to perform as individual robotic modules. At the microscale/nanoscale, controlling multiple robots is difficult due to the limitations of fabrication technologies and the availability of on-board controllers. This highlights the requirement of different approaches compared to macro systems for a group of microrobotic systems. Current microrobotic systems have the capability to form different configurations, either as a collectively actuated swarm or a selectively actuated group of agents. Magnetic, acoustic, electric, optical, and hybrid methods are reviewed under collective formation methods, and surface anchoring, heterogeneous design, and non-uniform control input are significant in the selective formation of microrobotic systems. In addition, actuation principles play an important role in designing microrobotic systems with multiple microrobots, and the various control systems are also reviewed because they affect the development of such systems at the microscale. Reconfigurability, self-adaptable motion, and enhanced imaging due to the aggregation of modules have shown potential applications specifically in the biomedical sector. This review presents the current state of shape formation using microrobots with regard to forming techniques, actuation principles, and control systems. Finally, the future developments of these systems are presented.

Nano/Micromotors in Active Matter

Nano/micromotors (NMMs) are tiny objects capable of converting energy into mechanical motion. Recently, a wealth of active matter including synthetic colloids, cytoskeletons, bacteria, and cells have been used to construct NMMs. The self-sustained motion of active matter drives NMMs out of equilibrium, giving rise to rich dynamics and patterns. Alongside the spontaneous dynamics, external stimuli such as geometric confinements, light, magnetic field, and chemical potential are also harnessed to control the movements of NMMs, yielding new application paradigms of active matter. Here, we review the recent advances, both experimental and theoretical, in exploring biological NMMs. The unique dynamical features of collective NMMs are focused on, along with some possible applications of these intriguing systems.

The control effort to steer self-propelled microswimmers depends on their morphology: comparing symmetric spherical versus asymmetric L-shaped particles

Active goal-directed motion requires real-time adjustment of control signals depending on the system's status, also known as control. The amount of information that needs to be processed depends on the desired motion and control, and on the system's morphology. The morphology of the system may directly effectuate or support the desired motion. This morphology-based reduction to the neuronal 'control effort' can be quantified by a novel information-entropy-based approach. Here, we apply this novel measure of 'control effort' to active microswimmers of different morphology. Their motion is a combination of directed deterministic and stochastic motion. In spherical microswimmers, the active propulsion leads to linear velocities. Active propulsion of asymmetric L-shaped particles leads to circular or-on tilted substrates-directed motion. Thus, the difference in shape, i.e. the morphology of the particles, directly influence the motion. Here, we quantify how this morphology can be exploited by control schemes for the purpose of steering the particles towards targets. Using computer simulations, we found in both cases a significantly lower control effort for L-shaped particles. However, certain movements can only be achieved by spherical particles. This demonstrates that a suitably designed microswimmer's morphology might be exploited to perform specific tasks.