A Novel Method for Intrinsic and Extrinsic Parameters Estimation by Solving Perspective-Three-Point Problem with Known Camera Position

The aim of the perspective-three-point (P3P) problem is to estimate extrinsic parameters of a camera from three 2D–3D point correspondences, including the orientation and position information. All the P3P solvers have a multi-solution phenomenon that is up to four solutions and needs a fully calibrated camera. In contrast, in this paper we propose a novel method for intrinsic and extrinsic parameter estimation based on three 2D–3D point correspondences with known camera position. Our core contribution is to build a new, virtual camera system whose frame and image plane are defined by the original 3D points, to build a new, intermediate world frame by the original image plane and the original 2D image points, and convert our problem to a P3P problem. Then, the intrinsic and extrinsic parameter estimation is to solve frame transformation and the P3P problem. Lastly, we solve the multi-solution problem by image resolution. Experimental results show its accuracy, numerical stability and uniqueness of the solution for intrinsic and extrinsic parameter estimation in synthetic data and real images.

Global Positioning from a Single Image of a Rectangle in Conical Perspective

This article presents a method to obtain the overall positioning of the focus of a camera from an image that includes a rectangle in a fixed reference with known position and dimension. This technique uses basic principles of descriptive geometry introduced in engineering courses. The document will first show how to obtain the dihedral projections of a rectangle after three turns and one translation. Secondly, we will proceed to obtain the image of the rectangle rotated in a conical perspective, taking the elevation plane as the drawing plane and a specific point in space as the view point, and represented in the dihedral system. Thirdly, we proceed with the inverse perspective transformation; we will expose a method to obtain the coordinates in the space of a rectangle obtained from an image. Finally, we check the method experimentally by taking an image of the rectangle with a camera in which the coordinates in the drawing plane (center of the image) are the only available position information. Then, the positioning and orientation of the camera in 3D will be obtained.

Visual Closed-Loop Dynamic Model Identification of Parallel Robots Based on Optical CMM Sensor

Parallel robots present outstanding advantages compared with their serial counterparts; they have both a higher force-to-weight ratio and better stiffness. However, the existence of closed-chain mechanism yields difficulties in designing control system for practical applications, due to its highly coupled dynamics. This paper focuses on the dynamic model identification of the 6-DOF parallel robots for advanced model-based visual servoing control design purposes. A visual closed-loop output-error identification method based on an optical coordinate-measuring-machine (CMM) sensor for parallel robots is proposed. The main advantage, compared with the conventional identification method, is that the joint torque measurement and the exact knowledge of the built-in robot controllers are not needed. The time-consuming forward kinematics calculation, which is employed in the conventional identification method of the parallel robot, can be avoided due to the adoption of optical CMM sensor for real time pose estimation. A case study on a 6-DOF RSS parallel robot is carried out in this paper. The dynamic model of the parallel robot is derived based on the virtual work principle, and the built dynamic model is verified through Matlab/SimMechanics. By using an outer loop visual servoing controller to stabilize both the parallel robot and the simulated model, a visual closed-loop output-error identification method is proposed and the model parameters are identified by using a nonlinear optimization technique. The effectiveness of the proposed identification algorithm is validated by experimental tests.

A method for going from 2D laparoscope to 3D acquisition of surface landmarks by a novel computer vision approach

PURPOSE

This paper presents a method to use the Smart Trocars-our new surgical instrument recognition system-or any accurate localization system of surgical instrument for acquiring intraoperative surface data. Complex laparoscopic surgeries need a proper guidance system which requires registering the preoperative data from a CT or MRI scan to the intraoperative patient state. The Smart Trocar can be used to localize the instruments when it comes to contact with the soft tissue surface.

METHOD

Two successive views through the laparoscope at different angles with the 3D localization of a fixed tool at one single location using the Smart Trocars can point out visible features during the surgery and acquire their location in 3D to provide a depth map in the region of interest. In other words, our method transforms a standard laparoscope system into a system with three-dimensional registration capability.

RESULT

This method was initially tested on a simulation for uncertainty assessment and then on a rigid model for verification with an accuracy within 2 mm distance. In addition, an in vivo experiment on pig model was also conducted to investigate how the method might be used during a physiologic respiratory cycle.

CONCLUSION

This method can be applied in a large number of surgical applications as a guidance system on its own or in conjunction with other navigation techniques. Our work encourages further testing with realistic surgical applications in the near future.

Pose Estimation from Line Correspondences: A Complete Analysis and a Series of Solutions

In this paper we deal with the camera pose estimation problem from a set of 2D/3D line correspondences, which is also known as PnL (Perspective-n-Line) problem. We carry out our study by comparing PnL with the well-studied PnP (Perspective-n-Point) problem, and our contributions are three-fold: (1) We provide a complete 3D configuration analysis for P3L, which includes the well-known P3P problem as well as several existing analyses as special cases. (2) By exploring the similarity between PnL and PnP, we propose a new subset-based PnL approach as well as a series of linear-formulation-based PnL approaches inspired by their PnP counterparts. (3) The proposed linear-formulation-based methods can be easily extended to deal with the line and point features simultaneously.

GoPros™ as an underwater photogrammetry tool for citizen science

Citizen science can increase the scope of research in the marine environment; however, it suffers from necessitating specialized training and simplified methodologies that reduce research output. This paper presents a simplified, novel survey methodology for citizen scientists, which combines GoPro imagery and structure from motion to construct an ortho-corrected 3D model of habitats for analysis. Results using a coral reef habitat were compared to surveys conducted with traditional snorkelling methods for benthic cover, holothurian counts, and coral health. Results were comparable between the two methods, and structure from motion allows the results to be analysed off-site for any chosen visual analysis. The GoPro method outlined in this study is thus an effective tool for citizen science in the marine environment, especially for comparing changes in coral cover or volume over time.

A position estimation system for mobile robots using a monocular image of a 3-D landmark

This paper presents an absolute position estimation system for a mobile robot moving on a flat surface. In this system, a 3-D landmark with four coplanar points and a non-coplanar point is utilized to improve the accuracy of position estimation and to guide the robot during navigation. Applying theoretical analysis, we investigate the image sensitivity of the proposed 3-D landmark compared with the conventional 2-D landmark. In the camera calibration stage of the experiments, we employ a neural network as a computational tool. The neural network is trained from a set of learning data collected at various points around the mark so that the extrinsic and intrinsic parameters of the camera system can be resolved. The overall estimation algorithm from the mark identification to the position determination is implemented in a 32-bit personal computer with an image digitizer and an arithmetic accelerator. To demonstrate the effectiveness of the proposed 3-D landmark and the neural network-based calibration scheme, a series of navigation experiments were performed on a wheeled mobile robot (LCAR) in an indoor environment. The results show the feasibility of the position estimation system applicable to mobile robot's real-time navigation.

Teleoperation with kinematically redundant robot manipulators with sub-task objectives

In this paper, control of nonlinear teleoperator systems where both the master and slave systems are kinematically redundant robot manipulators is addressed. The controller is developed under the assumption that the user and environmental input forces are unmeasurable. Lyapunov-based stability analysis is used to prove that the proposed controller yields asymptotic tracking results and ensures the coordination of the master and slave systems while satisfying a sub-task objective. Numerical simulation results are presented to illustrate the effectiveness of the proposed controller.

Automatic grasp planning for visual-servo controlled robotic manipulators

Vision based grasping holds great promise for grasping in dynamic environments where the object and/or robot are moving. The paper introduces a grasp planning approach for visual servo controlled robots with a single camera mounted at the end effector. Sensory control, mechanical, and geometrical issues in the design of such an automatic grasp planner are discussed and the corresponding constraints are highlighted. In particular, the integration of visual feature selection constraints is emphasized. Some quality measures are introduced to rate the candidate grasps. The grasp planning strategy and implementation issues in the development of an automatic grasp planner (AGP) are described. Simulation and experimental results are presented to verify the correctness of the approach and its effectiveness in dealing with dynamic situations.

Pose estimation in automated visual inspection using genetic algorithm

In this paper, we propose a genetic algorithm based approach to determine the pose of an object in Automated Visual Inspection having three degrees of freedom. We have investigated the effect of noise at 20 dB SNR and also mismatch resulting from incorrect correspondences between the object space points and the image space points, on the estimation of pose parameters. The maximum error in translation parameters is less than 0.45 cm and rotational error is less than 0.2 degree at 20 dB SNR. The error in parameter estimation is insignificant upto 7 pairs of mismatched points out of 24 points in object space and the results skyrockets when 8 or more pairs of points are mismatched. We have compared our result with that obtained by least square technique and it shows that GA based method outperform the gradient based technique when the number of vertices of the object to be inspected is small. These results have clearly established the robustness of GA in estimating the pose of an object with small number of vertices in automated visual inspection.

On pose recovery for generalized visual sensors

With the advances in imaging technologies for robot or machine vision, new imaging devices are being developed for robot navigation or image-based rendering. However, to satisfy some design criterion, such as image resolution or viewing ranges, these devices are not necessarily being designed to follow the perspective rule and, thus, the imaging rays may not pass through a common point. Such generalized imaging devices may not be perspective and, therefore, their poses cannot be estimated with traditional techniques. In this paper, we propose a systematic method for pose estimation of such a generalized imaging device. We formulate it as a nonperspective n point (NPnP) problem. The case with exact solutions, n=3, is investigated comprehensively. Approximate solutions can be found for n>3 in a least-squared-error manner by combining an initial-pose-estimation procedure and an orthogonally iterative procedure. This proposed method can be applied not only to nonperspective imaging devices but also perspective ones. Results from experiments show that our approach can solve the NPnP problem accurately.

Chronic whiplash associated disorders and neck movement measurements: an instantaneous helical axis approach

This paper presents an assessment tool for objective neck movement analysis of subjects suffering from chronic whiplash-associated disorders (WAD). Three-dimensional (3-D) motion data is collected by a commercially available motion analysis system. Head rotation, defined in this paper as the rotation angle around the instantaneous helical axis (IHA), is used for extracting a number of variables (e.g., angular velocity and range, symmetry of motion). Statistically significant differences were found between controls and subjects with chronic WAD in a number of variables.

Pose estimation in automated visual inspection using ANN

In this paper, we present an accurate and robust pose estimator of rigid, polyhedral objects, based on Artificial Neural Networks (ANN), as suitable for Automated Visual Inspection (AVI) applications. The estimator is novel in the sense that it is trained with different poses of the objects having dimensional deviations within its tolerance range and is therefore robust with respect to within tolerance dimensional errors. The estimation accuracy is scalable and our computer simulation experiments in the existing configurations of ANNs have shown an accuracy better than 4% of the placement error. The ANN based pose estimator offers several advantages over the classical implementations.