Multi-Cue Illumination Estimation via a Tree-Structured Group Joint Sparse Representation

Ranking-Based Color Constancy With Limited Training Samples

Computational color constancy is an important component of Image Signal Processors (ISP) for white balancing in many imaging devices. Recently, deep convolutional neural networks (CNN) have been introduced for color constancy. They achieve prominent performance improvements comparing with those statistics or shallow learning-based methods. However, the need for a large number of training samples, a high computational cost and a huge model size make CNN-based methods unsuitable for deployment on low-resource ISPs for real-time applications. In order to overcome these limitations and to achieve comparable performance to CNN-based methods, an efficient method is defined for selecting the optimal simple statistics-based method (SM) for each image. To this end, we propose a novel ranking-based color constancy method (RCC) that formulates the selection of the optimal SM method as a label ranking problem. RCC designs a specific ranking loss function, and uses a low rank constraint to control the model complexity and a grouped sparse constraint for feature selection. Finally, we apply the RCC model to predict the order of the candidate SM methods for a test image, and then estimate its illumination using the predicted optimal SM method (or fusing the results estimated by the top k SM methods). Comprehensive experiment results show that the proposed RCC outperforms nearly all the shallow learning-based methods and achieves comparable performance to (sometimes even better performance than) deep CNN-based methods with only 1/2000 of the model size and training time. RCC also shows good robustness to limited training samples and good generalization crossing cameras. Furthermore, to remove the dependence on the ground truth illumination, we extend RCC to obtain a novel ranking-based method without ground truth illumination (RCC_NO) that learns the ranking model using simple partial binary preference annotations provided by untrained annotators rather than experts. RCC_NO also achieves better performance than the SM methods and most shallow learning-based methods with low costs of sample collection and illumination measurement.

Color constancy with an optimized regularized random vector functional link based on an improved equilibrium optimizer

In order to improve the accuracy of illumination estimation, this paper proposes a color constancy algorithm based on an improved equilibrium optimizer (IEO) to optimize the network structure parameters and common parameters of the regularized random vector functional link (RRVFL) at the same time. First, the initial search agent (population) of the equilibrium optimizer algorithm is generated through opposition-based learning, and the particles (individuals in the population) of the search agent are updated using the IEO algorithm. Compared with the completely randomly generated search agent, the method of initializing the search agent through the IEO algorithm has a better convergence effect. Then, each segment of the search agent is mapped to the corresponding parameters of the RRVFL, and the effective input weight and hidden layer bias are selected according to the node activation to generate the network structure, which can realize the simultaneous optimization of hyperparameters and common parameters. Finally, by calculating the output weight, the light source color prediction of the image under unknown illumination is performed, and the image is corrected. Comparison experiments show that the IEO-RRVFL color constancy algorithm proposed in this paper has higher accuracy and better stability than other comparison algorithms.

Real-time factory smoke detection based on two-stage relation-guided algorithm

Recently, air quality analysis based on image sensing devices has attracted much attention. Since most smoke images in real scenes have challenging variances, which is difficult for existing object detection methods. To keep real-time factory smoke under efficient and universal social supervision, this paper proposes a mobile-platform-running efficient smoke detection algorithm based on image analysis techniques. We introduce the two-stage smoke detection (TSSD) algorithm based on the lightweight detection framework, in which the prior knowledge and contextual information are modeled into the relation-guided module to reduce the smoke search space, which can therefore significantly improve the performance of the single-stage method. Experimental results show that the proposed TSSD algorithm can robustly improve the detection accuracy of the single-stage method and the model has good compatibility for image resolution inputs. Compared with various state-of-the-art detection methods, the accuracy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$AP_{mean}$$\end{document}APmean of our proposed TSSD model reaches 59.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document}%, even surpassing the current detection model Faster RCNN. In addition, the detection speed of our proposed model can reach 50 ms (20 FPS), meeting the real-time requirements. This knowledge-based system has the advantages of high stability, high accuracy, fast detection speed. It can be widely used in some scenes with smoke detection requirements, such as on the mobile terminal carrier, providing great potential for practical environmental applications.

Which Features Are More Correlated to Illuminant Estimation: A Composite Substitute

Computational color constancy (CCC) is to endow computers or cameras with the capability to remove the color bias effect caused by different scene illuminations. The first procedure of CCC is illuminant estimation, i.e., to calculate the illuminant color for a given image scene. Recently, some methods directly mapping image features to illuminant estimation provide an effective and robust solution for this issue. Nevertheless, due to diverse image features, it is uncertain to select which features to model illuminant color. In this research, a series of artificial features weaved into a mapping-based illuminant estimation framework is extensively investigated. This framework employs a multi-model structure and integrates the functions of kernel-based fuzzy c-means (KFCM) clustering, non-negative least square regression (NLSR), and fuzzy weighting. By comparing the resulting performance of different features, the features more correlated to illuminant estimation are found in the candidate feature set. Furthermore, the composite features are designed to achieve the outstanding performances of illuminant estimation. Extensive experiments are performed on typical benchmark datasets and the effectiveness of the proposed method has been validated. The proposed method makes illuminant estimation an explicit transformation of suitable image features with regressed and fuzzy weights, which has significant potential for both competing performances and fast implementation against state-of-the-art methods.

Hybrid AI-assistive diagnostic model permits rapid TBS classification of cervical liquid-based thin-layer cell smears

Technical advancements significantly improve earlier diagnosis of cervical cancer, but accurate diagnosis is still difficult due to various factors. We develop an artificial intelligence assistive diagnostic solution, AIATBS, to improve cervical liquid-based thin-layer cell smear diagnosis according to clinical TBS criteria. We train AIATBS with >81,000 retrospective samples. It integrates YOLOv3 for target detection, Xception and Patch-based models to boost target classification, and U-net for nucleus segmentation. We integrate XGBoost and a logical decision tree with these models to optimize the parameters given by the learning process, and we develop a complete cervical liquid-based cytology smear TBS diagnostic system which also includes a quality control solution. We validate the optimized system with >34,000 multicenter prospective samples and achieve better sensitivity compared to senior cytologists, yet retain high specificity while achieving a speed of <180s/slide. Our system is adaptive to sample preparation using different standards, staining protocols and scanners.

Tracking droplets in soft granular flows with deep learning techniques

The state-of-the-art deep learning-based object recognition YOLO algorithm and object tracking DeepSORT algorithm are combined to analyze digital images from fluid dynamic simulations of multi-core emulsions and soft flowing crystals and to track moving droplets within these complex flows. The YOLO network was trained to recognize the droplets with synthetically prepared data, thereby bypassing the labor-intensive data acquisition process. In both applications, the trained YOLO + DeepSORT procedure performs with high accuracy on the real data from the fluid simulations, with low error levels in the inferred trajectories of the droplets and independently computed ground truth. Moreover, using commonly used desktop GPUs, the developed application is capable of analyzing data at speeds that exceed the typical image acquisition rates of digital cameras (30 fps), opening the interesting prospect of realizing a low-cost and practical tool to study systems with many moving objects, mostly but not exclusively, biological ones. Besides its practical applications, the procedure presented here marks the first step towards the automatic extraction of effective equations of motion of many-body soft flowing systems.

Combining Bottom-Up and Top-Down Visual Mechanisms for Color Constancy Under Varying Illumination

Multi-illuminant-based color constancy (MCC) is quite a challenging task. In this paper, we proposed a novel model motivated by the bottom-up and top-down mechanisms of human visual system (HVS) to estimate the spatially varying illumination in a scene. The motivation for bottom-up based estimation is from our finding that the bright and dark parts in a scene play different roles in encoding illuminants. However, handling the color shift of large colorful objects is difficult using pure bottom-up processing. Thus, we further introduce a top-down constraint inspired by the findings in visual psychophysics, in which high-level information (e.g., the prior of light source colors) plays a key role in visual color constancy. In order to implement the top-down hypothesis, we simply learn a color mapping between the illuminant distribution estimated by bottom-up processing and the ground truth maps provided by the dataset. We evaluated our model on four datasets and the results show that our method obtains very competitive performance compared with the state-of-the-art MCC algorithms. Moreover, the robustness of our model is more tangible considering that our results were obtained using the same parameters for all the datasets or the parameters of our model were learned from the inputs, that is, mimicking how HVS operates. We also show the color correction results on some real-world images taken from the web.

Improved single-illumination estimation accuracy via redefining the illuminant-invariant descriptor and the grey pixels

A grey pixel detection method, based on the illuminant-invariant descriptor in three logarithmic color channels, was proposed and proved effective for illuminant estimation [25]. In this work, an incremental improvement measure, based on Yang's work, is proposed by redefining the illuminant-invariant descriptor and the grey pixels. In order to improve the accuracy, appropriate parameters of grey pixels ratio and filter size are statistically analyzed and selected on different data sets. Experimental results show that the improved method is not only effective compared with classical statistics-based methods, but also outperforms many learning-based methods.

Improving Color Constancy in an Ambient Light Environment Using the Phong Reflection Model

We present a physics-based illumination estimation approach explicitly designed to handle natural images under ambient light. Existing physics-based color constancy methods are theoretically perfect but do not handle real-world images well because the majority of these methods assume a single illuminant. Therefore, specular pixels selected using existing methods produce estimated dichromatic lines that are thick or curvilinear in the presence of ambient light, thus generating significant errors. Based on the Phong reflection model, we show that a group of specular pixels on a uniformly colored object, although they are subject to intensity thresholding, produce a unique dichromatic line length depending on the geometry of each image path. Assuming that the longest dichromatic line is the most desirable when estimating the chromaticity of an illuminant, ambient-robust specular pixels are also found on the same path on which the longest dichromatic line segment is generated. Therefore, we propose a method to find the optimal image path in which the specular pixels produce the longest dichromatic line. Even though the number of collected specular pixels is reduced using the proposed method, they are proven to be more accurate when determining the illuminant chromaticity even in the existing methods. Experiments with an established benchmark data set and a self-produced image set find that the proposed method is better able to locate the illuminant chromaticity compared with the state-of-the-art color constancy methods.

Improving color constancy by discounting the variation of camera spectral sensitivity

It is an ill-posed problem to recover the true scene colors from a color biased image by discounting the effects of scene illuminant and camera spectral sensitivity (CSS) at the same time. Most color constancy (CC) models have been designed to first estimate the illuminant color, which is then removed from the color biased image to obtain an image taken under white light, without the explicit consideration of CSS effect on CC. This paper first studies the CSS effect on illuminant estimation arising in the inter-dataset-based CC (inter-CC), i.e., training a CC model on one dataset and then testing on another dataset captured by a distinct CSS. We show the clear degradation of existing CC models for inter-CC application. Then a simple way is proposed to overcome such degradation by first learning quickly a transform matrix between the two distinct CSSs (CSS-1 and CSS-2). The learned matrix is then used to convert the data (including the illuminant ground truth and the color-biased images) rendered under CSS-1 into CSS-2, and then train and apply the CC model on the color-biased images under CSS-2 without the need of burdensome acquiring of the training set under CSS-2. Extensive experiments on synthetic and real images show that our method can clearly improve the inter-CC performance for traditional CC algorithms. We suggest that, by taking the CSS effect into account, it is more likely to obtain the truly color constant images invariant to the changes of both illuminant and camera sensors.

The Reproduction Angular Error for Evaluating the Performance of Illuminant Estimation Algorithms

The angle between the RGBs of the measured illuminant and estimated illuminant colors-the recovery angular error-has been used to evaluate the performance of the illuminant estimation algorithms. However we noticed that this metric is not in line with how the illuminant estimates are used. Normally, the illuminant estimates are `divided out' from the image to, hopefully, provide image colors that are not confounded by the color of the light. However, even though the same reproduction results the same scene might have a large range of recovery errors. In this work the scale of the problem with the recovery error is quantified. Next we propose a new metric for evaluating illuminant estimation algorithms, called the reproduction angular error, which is defined as the angle between the RGB of a white surface when the actual and estimated illuminations are `divided out'. Our new metric ties algorithm performance to how the illuminant estimates are used. For a given algorithm, adopting the new reproduction angular error leads to different optimal parameters. Further the ranked list of best to worst algorithms changes when the reproduction angular is used. The importance of using an appropriate performance metric is established.

Tree-Structured Nuclear Norm Approximation with Applications to Robust Face Recognition

Structured sparsity, as an extension of standard sparsity, has shown the outstanding performance when dealing with some highly correlated variables in computer vision and pattern recognition. However, the traditional mixed (L1, L2) or (L1, L∞) group norm becomes weak in characterizing the internal structure of each group since they cannot alleviate the correla-tions between variables. Recently, nuclear norm has been vali-dated to be useful for depicting a spatially structured matrix variable. It considers the global structure of the matrix variable but overlooks the local structure. To combine the advantages of structured sparsity and nuclear norm, this paper presents a tree-structured nuclear norm approximation (TSNA) model as-suming that the representation residual with tree-structured prior is a random matrix variable and follows a dependent matrix dis-tribution. The Extended Alternating Direction Method of Multi-pliers (EADMM) is utilized to solve the proposed model. An effi-cient bound condition based on the extended restricted isometry constants is provided to show the exact recovery of the proposed model under the given noisy case. In addition, TSNA is connected with some newest methods such as sparse representation based classifier (SRC), nuclear-L1 norm joint regression (NL1R) and nuclear norm based matrix regression (NMR), which can be re-garded as the special cases of TSNA. Experiments with face re-construction and recognition demonstrate the benefits of TSNA over other approaches.