The Reproduction Angular Error for Evaluating the Performance of Illuminant Estimation Algorithms

Application of machine learning algorithms for accurate determination of bilirubin level on in vitro engineered tissue phantom images

Neonatal Jaundice is a common occurrence in neonates. High excess bilirubin would lead to hyperbilirubinemia, leading to irreversible adverse damage such as kernicterus. Therefore, it is necessary and important to monitor neonates' bilirubin levels in real-time for immediate intervention. However, current screening protocols have their inherent limitations, necessitating more convenient measurements. In this proof-of-concept study, we evaluated the feasibility of using machine learning for the screening of hyperbilirubinemia in neonates from smartphone-acquired photographs. Different machine learning models were compared and evaluated to gain a better understanding of feature selection and model performance in bilirubin determination. An in vitro study was conducted with a bilirubin-containing tissue phantom to identify potential biological and environmental confounding factors. The findings of this study present a systematic characterization of the confounding effect of various factors through separate parametric tests. These tests uncover potential techniques in image pre-processing, highlighting important biological features (light scattering property and skin thickness) and external features (ISO, lighting conditions and white balance), which together contribute to robust model approaches for accurately determining bilirubin concentrations. By obtaining an accuracy of 0.848 in classification and 0.812 in regression, these findings indicate strong potential in aiding in the design of clinical studies using patient-derived images.

Nighttime color constancy using robust gray pixels

Color constancy is a basic step for achieving stable color perception in both biological visual systems and the image signal processing (ISP) pipeline of cameras. So far, there have been numerous computational models of color constancy that focus on scenes under normal light conditions but are less concerned with nighttime scenes. Compared with daytime scenes, nighttime scenes usually suffer from relatively higher-level noise and insufficient lighting, which usually degrade the performance of color constancy methods designed for scenes under normal light. In addition, there is a lack of nighttime color constancy datasets, limiting the development of relevant methods. In this paper, based on the gray-pixel-based color constancy methods, we propose a robust gray pixel (RGP) detection method by carefully designing the computation of illuminant-invariant measures (IIMs) from a given color-biased nighttime image. In addition, to evaluate the proposed method, a new dataset that contains 513 nighttime images and corresponding ground-truth illuminants was collected. We believe this dataset is a useful supplement to the field of color constancy. Finally, experimental results show that the proposed method achieves superior performance to statistics-based methods. In addition, the proposed method was also compared with recent deep-learning methods for nighttime color constancy, and the results show the method's advantages in cross-validation among different datasets.

Color Constancy via Multi-Scale Region-Weighed Network Guided by Semantics

In obtaining color constancy, estimating the illumination of a scene is the most important task. However, due to unknown light sources and the influence of the external imaging environment, the estimated illumination is prone to color ambiguity. In this article, a learning-based multi-scale region-weighed network guided by semantic features is proposed to estimate the illuminated color of the light source in a scene. Cued by the human brain's processing of color constancy, we use image semantics and scale information to guide the process of illumination estimation. First, we put the image and its semantics into the network, and then obtain the region weights of the image at different scales. After that, through a special weight-pooling layer (WPL), the illumination on each scale is estimated. The final illumination is calculated by weighting each scale. The results of extensive experiments on Color Checker and NUS 8-Camera datasets show that the proposed approach is superior to the current state-of-the-art methods in both efficiency and effectiveness.

Which Features Are More Correlated to Illuminant Estimation: A Composite Substitute

Computational color constancy (CCC) is to endow computers or cameras with the capability to remove the color bias effect caused by different scene illuminations. The first procedure of CCC is illuminant estimation, i.e., to calculate the illuminant color for a given image scene. Recently, some methods directly mapping image features to illuminant estimation provide an effective and robust solution for this issue. Nevertheless, due to diverse image features, it is uncertain to select which features to model illuminant color. In this research, a series of artificial features weaved into a mapping-based illuminant estimation framework is extensively investigated. This framework employs a multi-model structure and integrates the functions of kernel-based fuzzy c-means (KFCM) clustering, non-negative least square regression (NLSR), and fuzzy weighting. By comparing the resulting performance of different features, the features more correlated to illuminant estimation are found in the candidate feature set. Furthermore, the composite features are designed to achieve the outstanding performances of illuminant estimation. Extensive experiments are performed on typical benchmark datasets and the effectiveness of the proposed method has been validated. The proposed method makes illuminant estimation an explicit transformation of suitable image features with regressed and fuzzy weights, which has significant potential for both competing performances and fast implementation against state-of-the-art methods.

Underwater Image Enhancement Using Adaptive Retinal Mechanisms

We propose an underwater image enhancement model inspired by the morphology and function of the teleost fish retina. We aim to solve the problems of underwater image degradation raised by the blurring and nonuniform color biasing. In particular, the feedback from color-sensitive horizontal cells to cones and a red channel compensation are used to correct the nonuniform color bias. The center-surround opponent mechanism of the bipolar cells and the feedback from amacrine cells to interplexiform cells then to horizontal cells serve to enhance the edges and contrasts of the output image. The ganglion cells with color-opponent mechanism are used for color enhancement and color correction. Finally, we adopt a luminance-based fusion strategy to reconstruct the enhanced image from the outputs of ON and OFF pathways of fish retina. Our model utilizes the global statistics (i.e., image contrast) to automatically guide the design of each low-level filter, which realizes the self-adaption of the main parameters. Extensive qualitative and quantitative evaluations on various underwater scenes validate the competitive performance of our technique. Our model also significantly improves the accuracy of transmission map estimation and local feature point matching using the underwater image. Our method is a single image approach that does not require the specialized prior about the underwater condition or scene structure.

Improving color constancy by selecting suitable set of training images

With very simple implementation, regression-based color constancy (CC) methods have recently obtained very competitive performance by applying a correction matrix to the results of some low level-based CC algorithms. However, most regression-based methods, e.g., Corrected Moment (CM), apply a same correction matrix to all the test images. Considering that the captured image color is usually determined by various factors (e.g., illuminant and surface reflectance), it is obviously not reasonable enough to apply a same correction to different test images without considering the intrinsic difference among images. In this work, we first mathematically analyze the key factors that may influence the performance of regression-based CC, and then we design principled rules to automatically select the suitable training images to learn an optimal correction matrix for each test image. With this strategy, the original regression-based CC (e.g., CM) is clearly improved to obtain more competitive performance on four widely used benchmark datasets. We also show that although this work focuses on improving the regression-based CM method, a noteworthy aspect of the proposed automatic training data selection strategy is its applicability to several representative regression-based approaches for the color constancy problem.

Multilevel weighted enhancement for underwater image dehazing

Attenuation and scattering of light are responsible for haziness in images of underwater scenes. To reduce this effect, we propose an approach for single-image dehazing by multilevel weighted enhancement of the image. The underlying principle is that enhancement at different levels of detail can undo the degradation caused by underwater haze. The depth information is captured implicitly while going through different levels of details due to the depth-variant nature of haze. Hence, we judiciously assign weights to different levels of image details and reveal that their linear combination along with the coarsest information can successfully restore the image. Results demonstrate the efficacy of our approach as compared to state-of-the-art underwater dehazing methods.

Colour Constancy for Image of Non-Uniformly Lit Scenes

Digital camera sensors are designed to record all incident light from a captured scene, but they are unable to distinguish between the colour of the light source and the true colour of objects. The resulting captured image exhibits a colour cast toward the colour of light source. This paper presents a colour constancy algorithm for images of scenes lit by non-uniform light sources. The proposed algorithm uses a histogram-based algorithm to determine the number of colour regions. It then applies the K-means++ algorithm on the input image, dividing the image into its segments. The proposed algorithm computes the Normalized Average Absolute Difference (NAAD) for each segment and uses it as a measure to determine if the segment has sufficient colour variations. The initial colour constancy adjustment factors for each segment with sufficient colour variation is calculated. The Colour Constancy Adjustment Weighting Factors (CCAWF) for each pixel of the image are determined by fusing the CCAWFs of the segments, weighted by their normalized Euclidian distance of the pixel from the center of the segments. Results show that the proposed method outperforms the statistical techniques and its images exhibit significantly higher subjective quality to those of the learning-based methods. In addition, the execution time of the proposed algorithm is comparable to statistical-based techniques and is much lower than those of the state-of-the-art learning-based methods.

Colour and illumination in computer vision

In computer vision, illumination is considered to be a problem that needs to be 'solved'. The colour cast due to illumination is removed to support colour-based image recognition and stable tracking (in and out of shadows), among other tasks. In this paper, I review historical and current algorithms for illumination estimation. In the classical approach, the illuminant colour is estimated by an ever more sophisticated analysis of simple image summary statistics often followed by a bias correction step. Bias correction is a function applied to the estimates made by a given illumination estimation algorithm to correct consistent errors in the estimations. Most recently, the full power, and much higher complexity, of deep learning has been deployed (where, effectively, the definition of the image statistics of interest and the type of analysis carried out are found as part of an overall optimization). In this paper, I challenge the orthodoxy of deep learning, i.e. that it is the best approach for illuminant estimation. We instead focus on the final bias correction stage found in many simple illumination estimation algorithms. There are two key insights in our method. First, we argue that the bias must be corrected in an exposure invariant way. Second, we show that this bias correction amounts to 'solving for a homography'. Homography-based illuminant estimation is shown to deliver leading illumination estimation performance (at a very small fraction of the complexity of deep learning methods).

Auto white balance by surface reflection decomposition

Auto white balance (AWB) is an important operation in color imaging applications. Most existing AWB algorithms rely on some physical features and statistical properties of natural scenes. However, the AWB algorithms using statistical properties are sensitive to the statistics of the scene contents. Therefore, it is highly desirable to find physical features that are more robust and relatively insensitive to scene contents. In this paper, we propose such physical features based on surface reflection decomposition. Light reflection from most object surfaces can be decomposed into a specular component and a diffuse component. Instead of trying to find the common axis of the color planes as in past algorithms, we estimate the illuminant chromaticity by searching through the light source candidates to find the one that will best cancel the specular components. We provide two formulations: the minimum projected area algorithm and the minimum total variation algorithm for estimation of the scene-illuminant chromaticity. Both show very favorable results compared with other published algorithms.