Variational model for simultaneously image denoising and contrast enhancement

Improved Calibration of Eye-in-Hand Robotic Vision System Based on Binocular Sensor

Eye-in-hand robotic binocular sensor systems are indispensable equipment in the modern manufacturing industry. However, because of the intrinsic deficiencies of the binocular sensor, such as the circle of confusion and observed error, the accuracy of the calibration matrix between the binocular sensor and the robot end is likely to decline. These deficiencies cause low accuracy of the matrix calibrated by the traditional method. In order to address this, an improved calibration method for the eye-in-hand robotic vision system based on the binocular sensor is proposed. First, to improve the accuracy of data used for solving the calibration matrix, a circle of confusion rectification method is proposed, which rectifies the position of the pixel in images in order to make the detected geometric feature close to the real situation. Subsequently, a transformation error correction method with the strong geometric constraint of a standard multi-target reference calibrator is developed, which introduces the observed error to the calibration matrix updating model. Finally, the effectiveness of the proposed method is validated by a series of experiments. The results show that the distance error is reduced to 0.080 mm from 0.192 mm compared with the traditional calibration method. Moreover, the measurement accuracy of local reference points with updated calibration results from the field is superior to 0.056 mm.

Robust contrast enhancement method using a retinex model with adaptive brightness for detection applications

Low light image enhancement with adaptive brightness, color and contrast preservation in degraded visual conditions (e.g., extreme dark background, lowlight, back-light, mist. etc.) is becoming more challenging for machine cognition applications than anticipated. A realistic image enhancement framework should preserve brightness and contrast in robust scenarios. The extant direct enhancement methods amplify objectionable structure and texture artifacts, whereas network-based enhancement approaches are based on paired or large-scale training datasets, raising fundamental concerns about their real-world applicability. This paper presents a new framework to get deep into darkness in degraded visual conditions following the fundamental of retinex-based image decomposition. We separate the reflection and illumination components to perform independent weighted enhancement operations on each component to preserve the visual details with a balance of brightness and contrast. A comprehensive weighting strategy is proposed to constrain image decomposition while disrupting the irregularities of high frequency reflection and illumination to improve the contrast. At the same time, we propose to guide the illumination component with a high-frequency component for structure and texture preservation in degraded visual conditions. Unlike existing approaches, the proposed method works regardless of the training data type (i.e., low light, normal light, or normal and low light pairs). A deep into darkness network (D2D-Net) is proposed to maintain the visual balance of smoothness without compromising the image quality. We conduct extensive experiments to demonstrate the superiority of the proposed enhancement. We test the performance of our method for object detection tasks in extremely dark scenarios. Experimental results demonstrate that our method maintains the balance of visual smoothness, making it more viable for future interactive visual applications.

CODEN: combined optimization-based decomposition and learning-based enhancement network for Retinex-based brightness and contrast enhancement

In this paper, we present a novel low-light image enhancement method by combining optimization-based decomposition and enhancement network for simultaneously enhancing brightness and contrast. The proposed method works in two steps including Retinex decomposition and illumination enhancement, and can be trained in an end-to-end manner. The first step separates the low-light image into illumination and reflectance components based on the Retinex model. Specifically, it performs model-based optimization followed by learning for edge-preserved illumination smoothing and detail-preserved reflectance denoising. In the second step, the illumination output from the first step, together with its gamma corrected and histogram equalized versions, serves as input to illumination enhancement network (IEN) including residual squeeze and excitation blocks (RSEBs). Extensive experiments prove that our method shows better performance compared with state-of-the-art low-light enhancement methods in the sense of both objective and subjective measures.

Stereo-IA: stereo visual intensity alignment and beyond under radiation variation

Stereo vision is a hot research topic at present, but due to the radiation changes, there will be a large intensity difference between stereo pairs, which will lead to serious degradation of stereo vision based matching, pose estimation, image segmentation and other tasks. Previous methods are not robust to radiation changes or have a large amount of calculation. Accordingly, this paper proposes a new stereo intensity alignment and image enhancement method based on the latest SuperPoint features. It combines the triangle based bearings-only metric, scale-ANCC and belief propagation model and has strong robustness to radiation changes. The quantitative and qualitative comparison experiments on Middlebury datasets verify the effectiveness of the proposed method, and it has a better image restoration and matching effect under the radiation changes.

A Novel Multi-Modality Image Simultaneous Denoising and Fusion Method Based on Sparse Representation

Multi-modality image fusion applied to improve image quality has drawn great attention from researchers in recent years. However, noise is actually generated in images captured by different types of imaging sensors, which can seriously affect the performance of multi-modality image fusion. As the fundamental method of noisy image fusion, source images are denoised first, and then the denoised images are fused. However, image denoising can decrease the sharpness of source images to affect the fusion performance. Additionally, denoising and fusion are processed in separate processing modes, which causes an increase in computation cost. To fuse noisy multi-modality image pairs accurately and efficiently, a multi-modality image simultaneous fusion and denoising method is proposed. In the proposed method, noisy source images are decomposed into cartoon and texture components. Cartoon-texture decomposition not only decomposes source images into detail and structure components for different image fusion schemes, but also isolates image noise from texture components. A Gaussian scale mixture (GSM) based sparse representation model is presented for the denoising and fusion of texture components. A spatial domain fusion rule is applied to cartoon components. The comparative experimental results confirm the proposed simultaneous image denoising and fusion method is superior to the state-of-the-art methods in terms of visual and quantitative evaluations.

Deep image enhancement for ill light imaging

Imaging in the natural scene under ill lighting conditions (e.g., low light, back-lit, over-exposed front-lit, and any combinations of them) suffers from both over- and under-exposure at the same time, whereas processing of such images often results in over- and under-enhancement. A single small image sensor can hardly provide satisfactory quality for ill lighting conditions with ordinary optical lenses in capturing devices. Challenges arise in the maintenance of a visual smoothness between those regions, while color and contrast should be well preserved. The problem has been approached by various methods, including multiple sensors and handcrafted parameters, but extant model capacity is limited to only some specific scenes (i.e., lighting conditions). Motivated by these challenges, in this paper, we propose a deep image enhancement method for color images captured under ill lighting conditions. In this method, input images are first decomposed into reflection and illumination maps with the proposed layer distribution loss net, where the illumination blindness and structure degradation problem can be subsequently solved via these two components, respectively. The hidden degradation in reflection and illumination is tuned with a knowledge-based adaptive enhancement constraint designed for ill illuminated images. The model can maintain a balance of smoothness and contribute to solving the problem of noise besides over- and under-enhancement. The local consistency in illumination is achieved via a repairing operation performed in the proposed Repair-Net. The total variation operator is optimized to acquire local consistency, and the image gradient is guided with the proposed enhancement constraint. Finally, a product of updated reflection and illumination maps reconstructs an enhanced image. Experiments are organized under both very low exposure and ill illumination conditions, where a new dataset is also proposed. Results on both experiments show that our method has superior performance in preserving structural and textural details compared to other states of the art, which suggests that our method is more practical in future visual applications.