Application of Motion-Based Visual Servoing to Target Tracking

In this paper, the classical task of mobile target tracking using a pan-and-tilt camera is considered. The authors use recent results in motion-based visual servoing to deal with complex targets for which shape and texture are unknown. The first method consists of designing a control law directly from the estimated image motion. This leads to the computation of the pan-and-tilt acceleration necessary to reduce the tracking error. A second method, more efficient for target tracking, consists of retrieving the target position in the image from its estimated motion. This leads to classical image-based visual servoing. For both methods, experimental results obtained at video rate are presented and discussed.

Building large mosaics of confocal edomicroscopic images using visual servoing

Probe-based confocal laser endomicroscopy provides real-time microscopic images of tissues contacted by a small probe that can be inserted in vivo through a minimally invasive access. Mosaicking consists in sweeping the probe in contact with a tissue to be imaged while collecting the video stream, and process the images to assemble them in a large mosaic. While most of the literature in this field has focused on image processing, little attention has been paid so far to the way the probe motion can be controlled. This is a crucial issue since the precision of the probe trajectory control drastically influences the quality of the final mosaic. Robotically controlled motion has the potential of providing enough precision to perform mosaicking. In this paper, we emphasize the difficulties of implementing such an approach. First, probe-tissue contacts generate deformations that prevent from properly controlling the image trajectory. Second, in the context of minimally invasive procedures targeted by our research, robotic devices are likely to exhibit limited quality of the distal probe motion control at the microscopic scale. To cope with these problems visual servoing from real-time endomicroscopic images is proposed in this paper. It is implemented on two different devices (a high-accuracy industrial robot and a prototype minimally invasive device). Experiments on different kinds of environments (printed paper and ex vivo tissues) show that the quality of the visually servoed probe motion is sufficient to build mosaics with minimal distortion in spite of disturbances.

Nonlinear visual mapping model for 3-D visual tracking with uncalibrated eye-in-hand robotic system

A new control scheme for uncalibrated robotic visual tracking problem is proposed that compromises the computational expenses of overall system with offline modeling and online control. A nonlinear visual mapping model for the uncalibrated hand-eye coordination is first proposed with an artificial neural network implementation. An online visual tracking controller is then developed together with a real-time motion planner. To improve the system performance, the control scheme is also integrated with a feedforward controller to compensate unknown object motions. Extensive simulations and experiments demonstrate the effectiveness of the proposed control scheme.