A perceptually inspired variational framework for color enhancement

Automated Identification of Hidden Corrosion Based on the D-Sight Technique: A Case Study on a Military Helicopter

Hidden corrosion remains a significant problem during aircraft service, primarily because of difficulties in its detection and assessment. The non-destructive D-Sight testing technique is characterized by high sensitivity to this type of damage and is an effective sensing tool for qualitative assessments of hidden corrosion in aircraft structures used by numerous ground service entities. In this paper, the authors demonstrated a new approach to the automatic quantification of hidden corrosion based on image processing D-Sight images during periodic inspections. The performance of the developed processing algorithm was demonstrated based on the results of the inspection of a Mi family military helicopter. The nondimensional quantitative measurement introduced in this study confirmed the effectiveness of this evaluation of corrosion progression, which was in agreement with the results of qualitative analysis of D-Sight images made by inspectors. This allows for the automation of the inspection process and supports inspectors in evaluating the extent and progression of hidden corrosion.

Learning Enriched Features for Fast Image Restoration and Enhancement

Given a degraded input image, image restoration aims to recover the missing high-quality image content. Numerous applications demand effective image restoration, e.g., computational photography, surveillance, autonomous vehicles, and remote sensing. Significant advances in image restoration have been made in recent years, dominated by convolutional neural networks (CNNs). The widely-used CNN-based methods typically operate either on full-resolution or on progressively low-resolution representations. In the former case, spatial details are preserved but the contextual information cannot be precisely encoded. In the latter case, generated outputs are semantically reliable but spatially less accurate. This paper presents a new architecture with a holistic goal of maintaining spatially-precise high-resolution representations through the entire network, and receiving complementary contextual information from the low-resolution representations. The core of our approach is a multi-scale residual block containing the following key elements: (a) parallel multi-resolution convolution streams for extracting multi-scale features, (b) information exchange across the multi-resolution streams, (c) non-local attention mechanism for capturing contextual information, and (d) attention based multi-scale feature aggregation. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Extensive experiments on six real image benchmark datasets demonstrate that our method, named as MIRNet-v2, achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement. The source code and pre-trained models are available at https://github.com/swz30/MIRNetv2.

Mathematical insights into the original Retinex algorithm for image enhancement

The Retinex theory, originally developed by Land and McCann as a computation model of the human color sensation, has become, with time, a pillar of digital image enhancement. In this area, the Retinex algorithm is widely used to improve the quality of any input image by increasing the visibility of its content and details, enhancing its colorfulness, and weakening, or even removing, some undesired effects of the illumination. The algorithm was originally described by its creators in terms of a sequence of image processing operations and was not fully formalized mathematically. Later, works focusing on aspects of the original formulation and adopting some of its principles tried to frame the algorithm within a mathematical formalism: this yielded every time a partial rendering of the model and resulted in several interesting model variants. The purpose of the present work is to fill a gap in the Retinex-related literature by providing a complete mathematical formalization of the original Retinex algorithm. The overarching goals of this work are to provide mathematical insights into the Retinex theory, promote awareness of the use of the model within image enhancement, and enable better appreciation of differences and similarities with later models based on Retinex principles. For this purpose, we compare our model with others proposed in the literature, paying particular attention to the work published in 2005 by Provenzi and others.

Contrast and Synthetic Multiexposure Fusion for Image Enhancement

Many hardware and software advancements have been made to improve image quality in smartphones, but unsuitable lighting conditions are still a significant impediment to image quality. To counter this problem, we present an image enhancement pipeline comprising synthetic multi-image exposure fusion and contrast enhancement robust to different lighting conditions. In this paper, we propose a novel technique of generating synthetic multi-exposure images by applying gamma correction to an input image using different values according to its luminosity for generating multiple intermediate images, which are then transformed into a final synthetic image by applying contrast enhancement. We observed that our proposed contrast enhancement technique focuses on specific regions of an image resulting in varying exposure, colors, and details for generating synthetic images. Visual and statistical analysis shows that our method performs better in various lighting scenarios and achieves better statistical naturalness and discrete entropy scores than state-of-the-art methods.

Retinex Based Image Enhancement via General Dictionary Convolutional Sparse Coding

Retinex theory represents the human visual system by showing the relative reflectance of an object under various illumination conditions. A feature of this human visual system is color constancy, and the Retinex theory is designed in consideration of this feature. The Retinex algorithms have been popularly used to effectively decompose the illumination and reflectance of an object. The main aim of this paper is to study image enhancement using convolution sparse coding and sparse representations of the reflectance component in the Retinex model over a learned dictionary. To realize this, we use the convolutional sparse coding model to represent the reflectance component in detail. In addition, we propose that the reflectance component can be reconstructed using a trained general dictionary by using convolutional sparse coding from a large dataset. We use singular value decomposition in limited memory to construct a best reflectance dictionary. This allows the reflectance component to provide improved visual quality over conventional methods, as shown in the experimental results. Consequently, we can reduce the difference in perception between humans and machines through the proposed Retinex-based image enhancement.

Endoscopic Vision Augmentation Using Multiscale Bilateral-Weighted Retinex for Robotic Surgery

Endoscopic vision plays a significant role in minimally invasive surgical procedures. The visibility and maintenance of such direct in situ vision is paramount not only for safety by preventing inadvertent injury but also to improve precision and reduce operating time. Unfortunately, the endoscopic vision is unavoidably degraded due to the illumination variations during surgery. This paper aims to restore or augment such degraded visualization and quantitatively evaluate it during robotic surgery. A multiscale bilateral-weighted retinex method is proposed to remove non-uniform and highly directional illumination and enhance surgical vision, while an objective no-reference image visibility assessment method is defined in terms of sharpness, naturalness, and contrast, to quantitatively and objectively evaluate the endoscopic visualization on surgical video sequences. The methods were validated on surgical data, with the experimental results showing that our method outperforms existent retinex approaches. In particular, the combined visibility was improved from 0.81 to 1.06, while three surgeons generally agreed that the results were restored with much better visibility.

Derivatives and inverse of cascaded linear+nonlinear neural models

In vision science, cascades of Linear+Nonlinear transforms are very successful in modeling a number of perceptual experiences. However, the conventional literature is usually too focused on only describing the forward input-output transform. Instead, in this work we present the mathematics of such cascades beyond the forward transform, namely the Jacobian matrices and the inverse. The fundamental reason for this analytical treatment is that it offers useful analytical insight into the psychophysics, the physiology, and the function of the visual system. For instance, we show how the trends of the sensitivity (volume of the discrimination regions) and the adaptation of the receptive fields can be identified in the expression of the Jacobian w.r.t. the stimulus. This matrix also tells us which regions of the stimulus space are encoded more efficiently in multi-information terms. The Jacobian w.r.t. the parameters shows which aspects of the model have bigger impact in the response, and hence their relative relevance. The analytic inverse implies conditions for the response and model parameters to ensure appropriate decoding. From the experimental and applied perspective, (a) the Jacobian w.r.t. the stimulus is necessary in new experimental methods based on the synthesis of visual stimuli with interesting geometrical properties, (b) the Jacobian matrices w.r.t. the parameters are convenient to learn the model from classical experiments or alternative goal optimization, and (c) the inverse is a promising model-based alternative to blind machine-learning methods for neural decoding that do not include meaningful biological information. The theory is checked by building and testing a vision model that actually follows a modular Linear+Nonlinear program. Our illustrative derivable and invertible model consists of a cascade of modules that account for brightness, contrast, energy masking, and wavelet masking. To stress the generality of this modular setting we show examples where some of the canonical Divisive Normalization modules are substituted by equivalent modules such as the Wilson-Cowan interaction model (at the V1 cortex) or a tone-mapping model (at the retina).

Color Contrast-Preserving Decolorization

Decolorization is to convert a color image into a gray scale image while preserve image features like salient structure and chrominance contrast. The sign of the color contrast is crucial for the decolorization algorithm and is usually determined in existing works by giving a strict defined color order or twomode weak order. In this paper, a fast computation on color order is achieved via a simple global mapping which is introduced in a linear parametric model using an extended structure transfer filter. The values of the parameters are obtained via an elegant approximation method. A local decolorization algorithm is finally designed on basis of the global linear mapping so that both color and spatial information are preserved robustly and accurately. Experimental results show that the proposed decolorization algorithms obtain a good performance among existing quality metrics for the decolorization. In addition, the proposed global decolorization algorithm is friendly to mobile devices with limited computational resource.

A population-based approach to point-sampling spatial color algorithms

Inspired by the behavior of the human visual system, spatial color algorithms perform image enhancement by correcting the pixel channel lightness based on the spatial distribution of the intensities in the surrounding area. The two visual contrast enhancement algorithms RSR and STRESS belong to this family of models: they rescale the input based on local reference values, which are determined by exploring the image by means of random point samples, called sprays. Due to the use of sampling, they may yield a noisy output. In this paper, we introduce a probabilistic formulation of the two models: our algorithms (RSR-P and STRESS-P) rely implicitly on the whole population of possible sprays. For processing larger images, we also provide two approximated algorithms that exploit a suitable target-dependent space quantization. Those spray population-based formulations outperform RSR and STRESS in terms of the processing time required for the production of noiseless outputs. We argue that this population-based approach, which can be extended to other members of the family, complements the sampling-based approach, in that it offers not only a better control in the design of approximated algorithms, but also additional insight into individual models and their relationships. We illustrate the latter point by providing a model of halo artifact formation.

Contrast Enhancement by Nonlinear Diffusion Filtering

To enhance the visual quality of an image that is degraded by uneven light, an effective method is to estimate the illumination component and compress it. Some previous methods have either defects of halo artifacts or contrast loss in the enhanced image due to incorrect estimation. In this paper, we discuss this problem and propose a novel method to estimate the illumination. The illumination is obtained by iteratively solving a nonlinear diffusion equation. During the diffusion process, surround suppression is embedded in the conductance function to specially enhance the diffusive strength in textural areas of the image. The proposed estimation method has the following two merits: 1) the boundary areas are preserved in the illumination, and thus halo artifacts are prevented and 2) the textural details are preserved in the reflectance to not suffer from illumination compression, which contributes to the contrast enhancement in the result. Experimental results show that the proposed algorithm achieves excellent performance in artifact removal and local contrast enhancement.

A Probabilistic Method for Image Enhancement With Simultaneous Illumination and Reflectance Estimation

In this paper, a new probabilistic method for image enhancement is presented based on a simultaneous estimation of illumination and reflectance in the linear domain. We show that the linear domain model can better represent prior information for better estimation of reflectance and illumination than the logarithmic domain. A maximum a posteriori (MAP) formulation is employed with priors of both illumination and reflectance. To estimate illumination and reflectance effectively, an alternating direction method of multipliers is adopted to solve the MAP problem. The experimental results show the satisfactory performance of the proposed method to obtain reflectance and illumination with visually pleasing enhanced results and a promising convergence rate. Compared with other testing methods, the proposed method yields comparable or better results on both subjective and objective assessments.

Fast Hue and Range Preserving Histogram: Specification: Theory and New Algorithms for Color Image Enhancement

Color image enhancement is a complex and challenging task in digital imaging with abundant applications. Preserving the hue of the input image is crucial in a wide range of situations. We propose simple image enhancement algorithms which conserve the hue and preserve the range (gamut) of the R, G, B channels in an optimal way. In our setup, the intensity input image is transformed into a target intensity image whose histogram matches a specified, well-behaved histogram. We derive a new color assignment methodology where the resulting enhanced image fits the target intensity image. We analyse the obtained algorithms in terms of chromaticity improvement and compare them with the unique and quite popular histogram based hue and range preserving algorithm of Naik and Murthy. Numerical tests confirm our theoretical results and show that our algorithms perform much better than the Naik-Murthy algorithm. In spite of their simplicity, they compete with well-established alternative methods for images where hue-preservation is desired.

From image processing to computational neuroscience: a neural model based on histogram equalization

There are many ways in which the human visual system works to reduce the inherent redundancy of the visual information in natural scenes, coding it in an efficient way. The non-linear response curves of photoreceptors and the spatial organization of the receptive fields of visual neurons both work toward this goal of efficient coding. A related, very important aspect is that of the existence of post-retinal mechanisms for contrast enhancement that compensate for the blurring produced in early stages of the visual process. And alongside mechanisms for coding and wiring efficiency, there is neural activity in the human visual cortex that correlates with the perceptual phenomenon of lightness induction. In this paper we propose a neural model that is derived from an image processing technique for histogram equalization, and that is able to deal with all the aspects just mentioned: this new model is able to predict lightness induction phenomena, and improves the efficiency of the representation by flattening both the histogram and the power spectrum of the image signal.

A no-reference perceptual based contrast enhancement metric for ocean scenes in fog

In this paper, we develop a perceptually based contrast enhancement metric as a means to solve the problem of autonomously enhancing images degraded by fog that are perceptually pleasing to humans. A learning based approach is considered to develop the contrast enhancement using human observations and low-level contrast enhancement metrics based on the human vision system. In addition, we provide new low-level metrics based on the physics of the scene to improve the performance of existing contrast enhancement metrics. This paper shows that a contrast enhancement metric can be designed to mimic human preference.

An Analysis of Visual Adaptation and Contrast Perception for Tone Mapping

Tone Mapping is the problem of compressing the range of a High-Dynamic Range image so that it can be displayed in a Low-Dynamic Range screen, without losing or introducing novel details: The final image should produce in the observer a sensation as close as possible to the perception produced by the real-world scene. We propose a tone mapping operator with two stages. The first stage is a global method that implements visual adaptation, based on experiments on human perception, in particular we point out the importance of cone saturation. The second stage performs local contrast enhancement, based on a variational model inspired by color vision phenomenology. We evaluate this method with a metric validated by psychophysical experiments and, in terms of this metric, our method compares very well with the state of the art.

A PDE formalization of Retinex theory

In 1964 Edwin H. Land formulated the Retinex theory, the first attempt to simulate and explain how the human visual system perceives color. His theory and an extension, the "reset Retinex" were further formalized by Land and McCann. Several Retinex algorithms have been developed ever since. These color constancy algorithms modify the RGB values at each pixel to give an estimate of the color sensation without a priori information on the illumination. Unfortunately, the Retinex Land-McCann original algorithm is both complex and not fully specified. Indeed, this algorithm computes at each pixel an average of a very large set of paths on the image. For this reason, Retinex has received several interpretations and implementations which, among other aims, attempt to tune down its excessive complexity. In this paper, it is proved that if the paths are assumed to be symmetric random walks, the Retinex solutions satisfy a discrete screened Poisson equation. This formalization yields an exact and fast implementation using only two FFTs. Several experiments on color images illustrate the effectiveness of the Retinex original theory.