Computer Vision on Mars

Information Visualization Based on Visual Transmission and Multimedia Data Fusion

With the rapid development of information technology, the application media of visual identity design has been greatly broadened, and the requirements of dynamic variability and interactivity of vision have become higher and higher. The introduction of data as an element has brought new semantic endowments to visual design. To solve the problems of noisy image reconstruction and many outliers in traditional art design, this paper uses an improved phase correlation algorithm to reconstruct multimedia video images. In addition, based on the visual characteristics of human eyes, a multi-feature fusion viewpoint image quality evaluation algorithm is proposed. The simulation results show that the method adopted in this paper improves the uneven image texture in art design, and its real effect is greatly enhanced.

The Use of Artificial Intelligence and Machine Learning in Surgery: A Comprehensive Literature Review

Interest in the use of artificial intelligence (AI) and machine learning (ML) in medicine has grown exponentially over the last few years. With its ability to enhance speed, precision, and efficiency, AI has immense potential, especially in the field of surgery. This article aims to provide a comprehensive literature review of artificial intelligence as it applies to surgery and discuss practical examples, current applications, and challenges to the adoption of this technology. Furthermore, we elaborate on the utility of natural language processing and computer vision in improving surgical outcomes, research, and patient care.

Field Programmable Gate Array Applications—A Scientometric Review

Field Programmable Gate Array (FPGA) is a general purpose programmable logic device that can be configured by a customer after manufacturing to perform from a simple logic gate operations to complex systems on chip or even artificial intelligence systems. Scientific publications related to FPGA started in 1992 and, up to now, we found more than 70,000 documents in the two leading scientific databases (Scopus and Clarivative Web of Science). These publications show the vast range of applications based on FPGAs, from the new mechanism that enables the magnetic suspension system for the kilogram redefinition, to the Mars rovers’ navigation systems. This paper reviews the top FPGAs’ applications by a scientometric analysis in ScientoPy, covering publications related to FPGAs from 1992 to 2018. Here we found the top 150 applications that we divided into the following categories: digital control, communication interfaces, networking, computer security, cryptography techniques, machine learning, digital signal processing, image and video processing, big data, computer algorithms and other applications. Also, we present an evolution and trend analysis of the related applications.

Descent trajectory reconstruction and landing site positioning of Chang'E-4 on the lunar farside

Chang'E-4 (CE-4) was the first mission to accomplish the goal of a successful soft landing on the lunar farside. The landing trajectory and the location of the landing site can be effectively reconstructed and determined using series of images obtained during descent when there were no Earth-based radio tracking and the telemetry data. Here we reconstructed the powered descent trajectory of CE-4 using photogrammetrically processed images of the CE-4 landing camera, navigation camera, and terrain data of Chang'E-2. We confirmed that the precise location of the landing site is 177.5991°E, 45.4446°S with an elevation of -5935 m. The landing location was accurately identified with lunar imagery and terrain data with spatial resolutions of 7 m/p, 5 m/p, 1 m/p, 10 cm/p and 5 cm/p. These results will provide geodetic data for the study of lunar control points, high-precision lunar mapping, and subsequent lunar exploration, such as by the Yutu-2 rover.

Vision-Based Behavior Planning for Lunar or Planetary Exploration Rover on Flat Surface

Lunar or planetary exploration rovers are expected to have the ability to move across an area as wide as possible in an unknown environment during a limited mission period. Hence, they need an efficient navigation method. Most of the surface of the moon or planets consists of flat ground, sand, and scattered rocks. In a simple flat sandy terrain with some rocks, rough route planning is sufficient for a lunar or planetary rover to avoid obstacles and reach an assigned point. This paper proposes an efficient vision-based planning scheme for exploration rovers on a flat surface with scattered obstacles. In the proposed scheme, dangerous areas are robustly extracted by processing image data, and the degree of danger is defined. A rough routing plan and sensing plan are simultaneously constructed based on the dangerous-area extraction results. The effectiveness of the proposed scheme is discussed based on the results of some simulations and simple experiments.

Optimization-based non-cooperative spacecraft pose estimation using stereo cameras during proximity operations

Pose estimation for spacecraft is widely recognized as an important technology for space applications. Many space missions require accurate relative pose between the chaser and the target spacecraft. Stereo-vision is a usual mean to estimate the pose of non-cooperative targets during proximity operations. However, the uncertainty of stereo-vision measurement is still an outstanding issue that needs to be solved. With binocular structure and the geometric structure of the object, we present a robust pose estimation method for non-cooperative spacecraft. Because the solar panel can provide strict geometry constraints, our approach takes the corner points of which as features. After stereo matching, an optimization-based method is proposed to estimate the relative pose between the two spacecraft. Simulation results show that our method improves the precision and robustness of pose estimation. Our system improves the performance with maximum 3D localization error less than 5% and relative rotation angle error less than 1°. Our laboratory experiments further validate the method.

Towards Camera-LIDAR Fusion-Based Terrain Modelling for Planetary Surfaces: Review and Analysis

In recent decades, terrain modelling and reconstruction techniques have increased research interest in precise short and long distance autonomous navigation, localisation and mapping within field robotics. One of the most challenging applications is in relation to autonomous planetary exploration using mobile robots. Rovers deployed to explore extraterrestrial surfaces are required to perceive and model the environment with little or no intervention from the ground station. Up to date, stereopsis represents the state-of-the art method and can achieve short-distance planetary surface modelling. However, future space missions will require scene reconstruction at greater distance, fidelity and feature complexity, potentially using other sensors like Light Detection And Ranging (LIDAR). LIDAR has been extensively exploited for target detection, identification, and depth estimation in terrestrial robotics, but is still under development to become a viable technology for space robotics. This paper will first review current methods for scene reconstruction and terrain modelling using cameras in planetary robotics and LIDARs in terrestrial robotics; then we will propose camera-LIDAR fusion as a feasible technique to overcome the limitations of either of these individual sensors for planetary exploration. A comprehensive analysis will be presented to demonstrate the advantages of camera-LIDAR fusion in terms of range, fidelity, accuracy and computation.

Integrated Navigation System Design for Micro Planetary Rovers: Comparison of Absolute Heading Estimation Algorithms and Nonlinear Filtering

This paper provides algorithms to fuse relative and absolute microelectromechanical systems (MEMS) navigation sensors, suitable for micro planetary rovers, to provide a more accurate estimation of navigation information, specifically, attitude and position. Planetary rovers have extremely slow speed (~1 cm/s) and lack conventional navigation sensors/systems, hence the general methods of terrestrial navigation may not be applicable to these applications. While relative attitude and position can be tracked in a way similar to those for ground robots, absolute navigation information is hard to achieve on a remote celestial body, like Moon or Mars, in contrast to terrestrial applications. In this study, two absolute attitude estimation algorithms were developed and compared for accuracy and robustness. The estimated absolute attitude was fused with the relative attitude sensors in a framework of nonlinear filters. The nonlinear Extended Kalman filter (EKF) and Unscented Kalman filter (UKF) were compared in pursuit of better accuracy and reliability in this nonlinear estimation problem, using only on-board low cost MEMS sensors. Experimental results confirmed the viability of the proposed algorithms and the sensor suite, for low cost and low weight micro planetary rovers. It is demonstrated that integrating the relative and absolute navigation MEMS sensors reduces the navigation errors to the desired level.

Autonomy for ground-level robotic space exploration: framework, simulation, architecture, algorithms and experiments

Robotic surface planetary exploration is a challenging endeavor, with critical safety requirements and severe communication constraints. Autonomous navigation is one of the most crucial and yet risky aspects of these operations. Therefore, a certain level of local autonomy for onboard robots is an essential feature, so that they can make their own decisions independently of ground control, reducing operational costs and maximizing the scientific return of the mission. In addition, existing tools to support research in this domain are usually proprietary to space agencies, and out of reach of most researchers. This paper presents a framework developed to support research in this field, a modular onboard software architecture design and a series of algorithms that implement a visual-based autonomous navigation approach for robotic space exploration. It allows analysis of algorithms' performance and functional validation of approaches and autonomy strategies, data monitoring and the creation of simulation models to replicate the vehicle, sensors, terrain and operational conditions. The framework and algorithms are partly supported by open-source packages and tools. A set of experiments and field testing with a physical robot and hardware are described as well, detailing results and algorithms' processing time, which experience an incremented of one order of magnitude when executed in space-certified like hardware, with constrained resources, in comparison to using general purpose hardware.

It's all about the constraints

Our ever-growing knowledge about the human brain and human behavior is opening doors to increasingly impressive technological achievements. This neurobiological inspiration has a significant history and involves almost equal parts neuroscience, computation and art. With a focus on the sense of vision, this essay presents a selective and highly condensed snapshot of the history of how neurobiology has inspired technological developments, pointing the way to where new inspirations may lead.

A survey on FPGA-based sensor systems: towards intelligent and reconfigurable low-power sensors for computer vision, control and signal processing

The current trend in the evolution of sensor systems seeks ways to provide more accuracy and resolution, while at the same time decreasing the size and power consumption. The use of Field Programmable Gate Arrays (FPGAs) provides specific reprogrammable hardware technology that can be properly exploited to obtain a reconfigurable sensor system. This adaptation capability enables the implementation of complex applications using the partial reconfigurability at a very low-power consumption. For highly demanding tasks FPGAs have been favored due to the high efficiency provided by their architectural flexibility (parallelism, on-chip memory, etc.), reconfigurability and superb performance in the development of algorithms. FPGAs have improved the performance of sensor systems and have triggered a clear increase in their use in new fields of application. A new generation of smarter, reconfigurable and lower power consumption sensors is being developed in Spain based on FPGAs. In this paper, a review of these developments is presented, describing as well the FPGA technologies employed by the different research groups and providing an overview of future research within this field.

Sky Region Detection in a Single Image for Autonomous Ground Robot Navigation

The sky region in an image provides horizontal and background information for autonomous ground robots and is important for vision-based autonomous ground robot navigation. This paper proposes a sky region detection algorithm within a single image based on gradient information and energy function optimization. Unlike most existing methods, the proposed algorithm is applicable to both colour and greyscale images. Firstly, the gradient information of the image is obtained. Then, the optimal segmentation threshold in the gradient domain is calculated according to the energy function optimization and the preliminary sky region is estimated. Finally, a post-processing method is applied in order to refine the preliminary sky region detection result when no sky region appears in the image or when objects extrude from the ground. Experimental results have proven that the detection accuracy is greater than 95% in our test set with 1,000 images, while the processing time is about 150ms for an image with a resolution of 640×480 on a modern laptop using only a single core.

Combined visual odometry and visual compass for off-road mobile robots localization

In this paper, we present the work related to the application of a visual odometry approach to estimate the location of mobile robots operating in off-road conditions. The visual odometry approach is based on template matching, which deals with estimating the robot displacement through a matching process between two consecutive images. Standard visual odometry has been improved using visual compass method for orientation estimation. For this purpose, two consumer-grade monocular cameras have been employed. One camera is pointing at the ground under the robot, and the other is looking at the surrounding environment. Comparisons with popular localization approaches, through physical experiments in off-road conditions, have shown the satisfactory behavior of the proposed strategy.

Field Testing of an Integrated Surface/Subsurface Modeling Technique for Planetary Exploration

While there has been much interest in developing ground-penetrating radar (GPR) technology for rover-based planetary exploration, relatively little work has been done on the data collection process. Starting from the manual method, we fully automate GPR data collection using only sensors typically found on a rover. Further, we produce two novel data products: (1) a three-dimensional, photorealistic surface model coupled with a ribbon of GPR data, and (2) a two-dimensional, topography-corrected GPR radargram with the surface topography plotted above. Each result is derived from only the onboard sensors of the rover, as would be required in a planetary exploration setting. These techniques were tested using data collected in a Mars analogue environment on Devon Island in the Canadian High Arctic. GPR transects were gathered over polygonal patterned ground similar to that seen on Mars by the Phoenix Lander. Using the techniques developed here, scientists may remotely explore the interaction of the surface topography and subsurface structure as if they were on site.

The improbability of harris interest points

An elementary characterization of the map underlying Harris corners, also known as Harris interest points or key points, is provided. Two principal and basic assumptions made are: 1) Local image structure is captured in an uncommitted way, simply using weighted raw image values around every image location to describe the local image information, and 2) the lower the probability of observing the image structure present in a particular point, the more salient, or interesting, this position is, i.e., saliency is related to how uncommon it is to see a certain image structure, how surprising it is. Through the latter assumption, the axiomatization proposed makes a sound link between image saliency in computer vision on the one hand and, on the other, computational models of preattentive human visual perception, where exactly the same definition of saliency has been proposed. Because of this link, the characterization provides a compelling case in favor of Harris interest points over other approaches.