Design and Control of 3-DoF Spherical Parallel Mechanism Robot Eyes Inspired by the Binocular Vestibule-ocular Reflex

The Design and Control of a Biomimetic Binocular Cooperative Perception System Inspired by the Eye Gaze Mechanism

Research on systems that imitate the gaze function of human eyes is valuable for the development of humanoid eye intelligent perception. However, the existing systems have some limitations, including the redundancy of servo motors, a lack of camera position adjustment components, and the absence of interest-point-driven binocular cooperative motion-control strategies. In response to these challenges, a novel biomimetic binocular cooperative perception system (BBCPS) was designed and its control was realized. Inspired by the gaze mechanism of human eyes, we designed a simple and flexible biomimetic binocular cooperative perception device (BBCPD). Based on a dynamic analysis, the BBCPD was assembled according to the principle of symmetrical distribution around the center. This enhances braking performance and reduces operating energy consumption, as evidenced by the simulation results. Moreover, we crafted an initial position calibration technique that allows for the calibration and adjustment of the camera pose and servo motor zero-position, to ensure that the state of the BBCPD matches the subsequent control method. Following this, a control method for the BBCPS was developed, combining interest point detection with a motion-control strategy. Specifically, we propose a binocular interest-point extraction method based on frequency-tuned and template-matching algorithms for perceiving interest points. To move an interest point to a principal point, we present a binocular cooperative motion-control strategy. The rotation angles of servo motors were calculated based on the pixel difference between the principal point and the interest point, and PID-controlled servo motors were driven in parallel. Finally, real experiments validated the control performance of the BBCPS, demonstrating that the gaze error was less than three pixels.

Gaze Point Tracking Based on a Robotic Body-Head-Eye Coordination Method

When the magnitude of a gaze is too large, human beings change the orientation of their head or body to assist their eyes in tracking targets because saccade alone is insufficient to keep a target at the center region of the retina. To make a robot gaze at targets rapidly and stably (as a human does), it is necessary to design a body-head-eye coordinated motion control strategy. A robot system equipped with eyes and a head is designed in this paper. Gaze point tracking problems are divided into two sub-problems: in situ gaze point tracking and approaching gaze point tracking. In the in situ gaze tracking state, the desired positions of the eye, head and body are calculated on the basis of minimizing resource consumption and maximizing stability. In the approaching gaze point tracking state, the robot is expected to approach the object at a zero angle. In the process of tracking, the three-dimensional (3D) coordinates of the object are obtained by the bionic eye and then converted to the head coordinate system and the mobile robot coordinate system. The desired positions of the head, eyes and body are obtained according to the object's 3D coordinates. Then, using sophisticated motor control methods, the head, eyes and body are controlled to the desired position. This method avoids the complex process of adjusting control parameters and does not require the design of complex control algorithms. Based on this strategy, in situ gaze point tracking and approaching gaze point tracking experiments are performed by the robot. The experimental results show that body-head-eye coordination gaze point tracking based on the 3D coordinates of an object is feasible. This paper provides a new method that differs from the traditional two-dimensional image-based method for robotic body-head-eye gaze point tracking.

3D-Printed Bionic Ear for Sound Identification and Localization Based on In Situ Polling of PVDF-TrFE Film

Bionic acoustic sensors are an indispensable part to realize interactions between humans and robotics. In this work, a PVDF-TrFE sensor array with multiple active pixels combined with a 3D-printed bionic ear model is prepared, which can accurately detect sounds with different frequencies and locate the sound source from different directions. The PVDF-TrFE sensor array can clearly identify the sound within 25 cm, and the error between the accepted sound frequency and the original input frequency is less than 0.001%. Through the algorithm analysis of the input signal, the location of the sound source can be immediately analyzed. Compared with other acoustic sensors, this sensor has the advantages of being self-powered, small size, and high flexibility, which holds great potential for bionic applications.

Enhancing the Sense of Attention from an Assistance Mobile Robot by Improving Eye-Gaze Contact from Its Iconic Face Displayed on a Flat Screen

One direct way to express the sense of attention in a human interaction is through the gaze. This paper presents the enhancement of the sense of attention from the face of a human-sized mobile robot during an interaction. This mobile robot was designed as an assistance mobile robot and uses a flat screen at the top of the robot to display an iconic (simplified) face with big round eyes and a single line as a mouth. The implementation of eye-gaze contact from this iconic face is a problem because of the difficulty of simulating real 3D spherical eyes in a 2D image considering the perspective of the person interacting with the mobile robot. The perception of eye-gaze contact has been improved by manually calibrating the gaze of the robot relative to the location of the face of the person interacting with the robot. The sense of attention has been further enhanced by implementing cyclic face explorations with saccades in the gaze and by performing blinking and small movements of the mouth.

Two degree-of-freedom robotic eye: design, modeling, and learning-based control in foveation and smooth pursuit

With increasing ocular motility disorders affecting human eye movement, the need to understand the biomechanics of the human eye rises constantly. A robotic eye system that physically mimics the human eye can serve as a useful tool for biomedical researchers to obtain an intuitive understanding of the functions and defects of the extraocular muscles and the eye. This paper presents the design, modeling, and control of a two degree-of-freedom (2-DOF) robotic eye, driven by artificial muscles, in particular, made of super-coiled polymers (SCPs). Considering the highly nonlinear dynamics of the robotic eye system, this paper applies deep deterministic policy gradient (DDPG), a machine learning algorithm to solve the control design problem in foveation and smooth pursuit of the robotic eye. To the best of our knowledge, this paper presents the first modeling effort to establish the dynamics of a robotic eye driven by SCP actuators, as well as the first control design effort for robotic eyes using a DDPG-based control strategy. A linear quadratic regulator-type reward function is proposed to achieve a balance between system performances (convergence speed and tracking accuracy) and control efforts. Simulation results are presented to demonstrate the effectiveness of the proposed control strategy for the 2-DOF robotic eye.