An SVD-based grayscale image quality measure for local and global assessment

Underwater single photon 3D imaging with millimeter depth accuracy and reduced blind range

Mono-static system benefits from its more flexible field of view and simplified structure, however, the backreflection photons from mono-static system lead to count loss for target detection. Counting loss engender range-blind, impeding the accurate acquisition of target depth. In this paper, count loss is reduced by introducing a polarization-based underwater mono-static single-photon imaging method, and hence reduced blind range. The proposed method exploits the polarization characteristic of light to effectively reduce the count loss of the target, thus improving the target detection efficiency. Experiments demonstrate that the target profile can be visually identified under our method, while the unpolarization system can not. Moreover, the ranging precision of system reaches millimeter-level. Finally, the target profile is reconstructed using non-local pixel correlations algorithm.

Using HVS Dual-Pathway and Contrast Sensitivity to Blindly Assess Image Quality

Blind image quality assessment (BIQA) aims to evaluate image quality in a way that closely matches human perception. To achieve this goal, the strengths of deep learning and the characteristics of the human visual system (HVS) can be combined. In this paper, inspired by the ventral pathway and the dorsal pathway of the HVS, a dual-pathway convolutional neural network is proposed for BIQA tasks. The proposed method consists of two pathways: the "what" pathway, which mimics the ventral pathway of the HVS to extract the content features of distorted images, and the "where" pathway, which mimics the dorsal pathway of the HVS to extract the global shape features of distorted images. Then, the features from the two pathways are fused and mapped to an image quality score. Additionally, gradient images weighted by contrast sensitivity are used as the input to the "where" pathway, allowing it to extract global shape features that are more sensitive to human perception. Moreover, a dual-pathway multi-scale feature fusion module is designed to fuse the multi-scale features of the two pathways, enabling the model to capture both global features and local details, thus improving the overall performance of the model. Experiments conducted on six databases show that the proposed method achieves state-of-the-art performance.

Visual stream connectivity predicts assessments of image quality

Despite extensive study of early vision, new and unexpected mechanisms continue to be identified. We introduce a novel formal treatment of the psychophysics of image similarity, derived directly from straightforward connectivity patterns in early visual pathways. The resulting differential geometry formulation is shown to provide accurate and explanatory accounts of human perceptual similarity judgments. The direct formal predictions are then shown to be further improved via simple regression on human behavioral reports, which in turn are used to construct more elaborate hypothesized neural connectivity patterns. It is shown that the predictive approaches introduced here outperform a standard successful published measure of perceived image fidelity; moreover, the approach provides clear explanatory principles of these similarity findings.

Two-dimensional TIRF-SIM-traction force microscopy (2D TIRF-SIM-TFM)

Quantifying small, rapidly evolving forces generated by cells is a major challenge for the understanding of biomechanics and mechanobiology in health and disease. Traction force microscopy remains one of the most broadly applied force probing technologies but typically restricts itself to slow events over seconds and micron-scale displacements. Here, we improve >2-fold spatially and >10-fold temporally the resolution of planar cellular force probing compared to its related conventional modalities by combining fast two-dimensional total internal reflection fluorescence super-resolution structured illumination microscopy and traction force microscopy. This live-cell 2D TIRF-SIM-TFM methodology offers a combination of spatio-temporal resolution enhancement relevant to forces on the nano- and sub-second scales, opening up new aspects of mechanobiology to analysis.

The Influence of the Window Width on FRA Assessment with Numerical Indices

Frequency response analysis is a method used in transformer diagnostics for the detection of mechanical faults or short-circuits in windings. The interpretation of test results is often performed with the application of numerical indices. However, usually these indices are used for the whole frequency range of the recorded data, returning a single number. Such an approach is inaccurate and may lead to mistakes in the interpretation. An alternative quality assessment is based on the estimation of the local values of the quality index with the moving window method. In this paper, the authors analyse the influence of the width of the input data window for four numerical indices. The analysis is based on the data measured on the transformer with deformations introduced into the winding and also for a 10 MVA transformer measured under industrial conditions. For the first unit the analysis is performed for various window widths and for various extents of the deformation, while in the case of the second the real differences between the frequency response curves are being analysed. On the basis of the results it was found that the choice of the data window width significantly influences the quality of the analysis results and the rules for elements number selection differ for various numerical indices.

A Singular Value Decomposition based Maximal Poisson-disk Sampling for adaptive Digital Elevation Model simplification

The proposed method is to do simplification for Digital Elevation Model (DEM), which uses a few of original nodes representing the terrain surface while maintaining the accuracy. The original DEM nodes are sampled using the Maximal Poisson-disk Sampling (MPS), in which, the disk's size of each sample is computed on basis of the Singular Value Decomposition (SVD). MPS can generate the hyper-uniformly distributed samples and was taken to do DEM adaptive sampling by being combined with the geodesic metric. However, the geodesic distance computation is complex and the requirement for memory is high. As such, this paper proposes an extension of the classic MPS based method for selecting quasi-randomly distributed points from DEM nodes based on the distribution of eigenvalues, accounting for surface heterogeneity. To achieve this objective, uniform MPS is conducted to sample the DEM nodes by setting the related disk radius to be inversely proportional to the local terrain complexity, which is defined as an index expressing the local terrain variation. Then, the geodesic metric related parameters are implicitly contained in the defined index. As a result, more samples are concentrated in the rugged regions, and vice versa. The proposed method shows better perfermance, at least the results are comparable with the geodesic distance based Poisson disk sampling method. Meanwhile, it greatly accelerates the sampling process and reduces the memory cost.

Full-Reference Quality Metric Based on Neural Network to Assess the Visual Quality of Remote Sensing Images

Remote sensing images are subject to different types of degradations. The visual quality of such images is important because their visual inspection and analysis are still widely used in practice. To characterize the visual quality of remote sensing images, the use of specialized visual quality metrics is desired. Although the attempts to create such metrics are limited, there is a great number of visual quality metrics designed for other applications. Our idea is that some of these metrics can be employed in remote sensing under the condition that those metrics have been designed for the same distortion types. Thus, image databases that contain images with types of distortions that are of interest should be looked for. It has been checked what known visual quality metrics perform well for images with such degradations and an opportunity to design neural network-based combined metrics with improved performance has been studied. It is shown that for such combined metrics, their Spearman correlation coefficient with mean opinion score exceeds 0.97 for subsets of images in the Tampere Image Database (TID2013). Since different types of elementary metric pre-processing and neural network design have been considered, it has been demonstrated that it is enough to have two hidden layers and about twenty inputs. Examples of using known and designed visual quality metrics in remote sensing are presented.

Quality Prediction on Deep Generative Images

In recent years, deep neural networks have been utilized in a wide variety of applications including image generation. In particular, generative adversarial networks (GANs) are able to produce highly realistic pictures as part of tasks such as image compression. As with standard compression, it is desirable to be able to automatically assess the perceptual quality of generative images to monitor and control the encode process. However, existing image quality algorithms are ineffective on GAN generated content, especially on textured regions and at high compressions. Here we propose a new "naturalness"-based image quality predictor for generative images. Our new GAN picture quality predictor is built using a multi-stage parallel boosting system based on structural similarity features and measurements of statistical similarity. To enable model development and testing, we also constructed a subjective GAN image quality database containing (distorted) GAN images and collected human opinions of them. Our experimental results indicate that our proposed GAN IQA model delivers superior quality predictions on the generative image datasets, as well as on traditional image quality datasets.

Frequency Response Quality Index for Assessing the Mechanical Condition of Transformer Windings

Frequency response analysis (FRA) is a popular method for assessing a transformer’s mechanical condition. The paper proposes a new method for interpreting the frequency response measurement results. The currently used numerical indices only give one value, which may be misleading in the analysis, while the proposed frequency response quality index (FRQI) tool analyses three separate features in the whole frequency range. The applied numerical calculations technique allows for estimations of not only the values of the average quality indices, but also locally for given frequency ranges of the analysed spectrum. It allows for determination of the problems that can be found in the active part of a transformer. The presented results come from three transformers, representing cases of typical faults. Two of them are from industry, while one was used for deformational tests in laboratory conditions. The proposed FRQI method showed its usefulness in FRA test results analysis and may be introduced into the automated assessment of such data. Each of the component parameters is sensitive to other types of differences observed between the compared frequency response curves, and may be used as a good quality detection tool.

Image Compression Based on Block SVD Power Method

In recent years, the important and fast growth in the development and demand of multimedia products is contributing to an insufficiency in the bandwidth of devices and network storage memory. Consequently, the theory of data compression becomes more significant for reducing data redundancy in order to allow more transfer and storage of data. In this context, this paper addresses the problem of lossy image compression. Indeed, this new proposed method is based on the block singular value decomposition (SVD) power method that overcomes the disadvantages of MATLAB’s SVD function in order to make a lossy image compression. The experimental results show that the proposed algorithm has better compression performance compared with the existing compression algorithms that use MATLAB’s SVD function. In addition, the proposed approach is simple in terms of implementation and can provide different degrees of error resilience, which gives, in a short execution time, a better image compression.

A Review on the Extraction of Quantitative Retinal Microvascular Image Feature

Digital image processing is one of the most widely used computer vision technologies in biomedical engineering. In the present modern ophthalmological practice, biomarkers analysis through digital fundus image processing analysis greatly contributes to vision science. This further facilitates developments in medical imaging, enabling this robust technology to attain extensive scopes in biomedical engineering platform. Various diagnostic techniques are used to analyze retinal microvasculature image to enable geometric features measurements such as vessel tortuosity, branching angles, branching coefficient, vessel diameter, and fractal dimension. These extracted markers or characterized fundus digital image features provide insights and relates quantitative retinal vascular topography abnormalities to various pathologies such as diabetic retinopathy, macular degeneration, hypertensive retinopathy, transient ischemic attack, neovascular glaucoma, and cardiovascular diseases. Apart from that, this noninvasive research tool is automated, allowing it to be used in large-scale screening programs, and all are described in this present review paper. This paper will also review recent research on the image processing-based extraction techniques of the quantitative retinal microvascular feature. It mainly focuses on features associated with the early symptom of transient ischemic attack or sharp stroke.

Comparison-Based Image Quality Assessment for Selecting Image Restoration Parameters

Image quality assessment (IQA) is traditionally classified into full-reference (FR) IQA, reduced-reference (RR) IQA, and no-reference (NR) IQA according to the amount of information required from the original image. Although NR-IQA and RR-IQA are widely used in practical applications, room for improvement still remains because of the lack of the reference image. Inspired by the fact that in many applications, such as parameter selection for image restoration algorithms, a series of distorted images are available, the authors propose a novel comparison-based IQA (C-IQA) framework. The new comparison-based framework parallels FR-IQA by requiring two input images and resembles NR-IQA by not using the original image. As a result, the new comparison-based approach has more application scenarios than FR-IQA does, and takes greater advantage of the accessible information than the traditional single-input NR-IQA does. Further, C-IQA is compared with other state-of-the-art NR-IQA methods and another RR-IQA method on two widely used IQA databases. Experimental results show that C-IQA outperforms the other methods for parameter selection, and the parameter trimming framework combined with C-IQA saves the computation of iterative image reconstruction up to 80%.

Image quality assessment based on inter-patch and intra-patch similarity

In this paper, we propose a full-reference (FR) image quality assessment (IQA) scheme, which evaluates image fidelity from two aspects: the inter-patch similarity and the intra-patch similarity. The scheme is performed in a patch-wise fashion so that a quality map can be obtained. On one hand, we investigate the disparity between one image patch and its adjacent ones. This disparity is visually described by an inter-patch feature, where the hybrid effect of luminance masking and contrast masking is taken into account. The inter-patch similarity is further measured by modifying the normalized correlation coefficient (NCC). On the other hand, we also attach importance to the impact of image contents within one patch on the IQA problem. For the intra-patch feature, we consider image curvature as an important complement of image gradient. According to local image contents, the intra-patch similarity is measured by adaptively comparing image curvature and gradient. Besides, a nonlinear integration of the inter-patch and intra-patch similarity is presented to obtain an overall score of image quality. The experiments conducted on six publicly available image databases show that our scheme achieves better performance in comparison with several state-of-the-art schemes.

Ranks for pairs of spatial fields via metric based on grayscale morphological distances

Based on a set of morphological distances computed between the grayscale images (spatial fields) of similar size specifications, the ratios of selected morphological distances, and the ratios of areas of infima and suprema of grayscale images, a new metric to quantify the degree of similarity between the grayscale images is proposed. We denote the two spatial fields (grayscale images), respectively, with f(i) and f(j), and the infima and suprema of these spatial fields with (f(i)∧f(j)) and ( f(i)∨f(j)). The three morphology-based distances include: 1) dilation distance d( f(i),f(j)) ; 2) erosion distance e( f(i),f(j)); and 3) median-based distance MN ( f(i),f(j)) . By employing these parameters, which play vital role in construction of parameter-specific interaction matrices, we provide a metric to designate every possible pair of images that can be considered out of a database consisting of a huge number of images. We demonstrate the whole approach on: 1) synthetic spatial fields; 2) a set of 12 similar-sized grayscale images representing cloud-top temperatures of a specific region for 12 different time instants; and 3) four spatial elevation fields to rank possible pairs of images.

Blind image blur assessment using singular value similarity and blur comparisons

The increasing number of demanding consumer image applications has led to increased interest in no-reference objective image quality assessment (IQA) algorithms. In this paper, we propose a new blind blur index for still images based on singular value similarity. The algorithm consists of three steps. First, a re-blurred image is produced by applying a Gaussian blur to the test image. Second, a singular value decomposition is performed on the test image and re-blurred image. Finally, an image blur index is constructed based on singular value similarity. The experimental results obtained on four simulated databases to demonstrate that the proposed algorithm has high correlation with human judgment when assessing blur or noise distortion of images.

Seven Challenges in Image Quality Assessment: Past, Present, and Future Research

Image quality assessment (IQA) has been a topic of intense research over the last several decades. With each year comes an increasing number of new IQA algorithms, extensions of existing IQA algorithms, and applications of IQA to other disciplines. In this article, I first provide an up-to-date review of research in IQA, and then I highlight several open challenges in this field. The first half of this article provides discuss key properties of visual perception, image quality databases, existing full-reference, no-reference, and reduced-reference IQA algorithms. Yet, despite the remarkable progress that has been made in IQA, many fundamental challenges remain largely unsolved. The second half of this article highlights some of these challenges. I specifically discuss challenges related to lack of complete perceptual models for: natural images, compound and suprathreshold distortions, and multiple distortions, and the interactive effects of these distortions on the images. I also discuss challenges related to IQA of images containing nontraditional, and I discuss challenges related to the computational efficiency. The goal of this article is not only to help practitioners and researchers keep abreast of the recent advances in IQA, but to also raise awareness of the key limitations of current IQA knowledge.

Fourier transform based scalable image quality measure

We present a new image quality assessment (IQA) algorithm based on the phase and magnitude of the 2D (twodimensional) Discrete Fourier Transform (DFT). The basic idea is to compare the phase and magnitude of the reference and distorted images to compute the quality score. However, it is well known that the Human Visual Systems (HVSs) sensitivity to different frequency components is not the same. We accommodate this fact via a simple yet effective strategy of nonuniform binning of the frequency components. This process also leads to reduced space representation of the image thereby enabling the reduced-reference (RR) prospects of the proposed scheme. We employ linear regression to integrate the effects of the changes in phase and magnitude. In this way, the required weights are determined via proper training and hence more convincing and effective. Lastly, using the fact that phase usually conveys more information than magnitude, we use only the phase for RR quality assessment. This provides the crucial advantage of further reduction in the required amount of reference image information. The proposed method is therefore further scalable for RR scenarios. We report extensive experimental results using a total of 9 publicly available databases: 7 image (with a total of 3832 distorted images with diverse distortions) and 2 video databases (totally 228 distorted videos). These show that the proposed method is overall better than several of the existing fullreference (FR) algorithms and two RR algorithms. Additionally, there is a graceful degradation in prediction performance as the amount of reference image information is reduced thereby confirming its scalability prospects. To enable comparisons and future study, a Matlab implementation of the proposed algorithm is available at http://www.ntu.edu.sg/home/wslin/reduced_phase.rar.

Image quality assessment based on gradient similarity

In this paper, we propose a new image quality assessment (IQA) scheme, with emphasis on gradient similarity. Gradients convey important visual information and are crucial to scene understanding. Using such information, structural and contrast changes can be effectively captured. Therefore, we use the gradient similarity to measure the change in contrast and structure in images. Apart from the structural/contrast changes, image quality is also affected by luminance changes, which must be also accounted for complete and more robust IQA. Hence, the proposed scheme considers both luminance and contrast-structural changes to effectively assess image quality. Furthermore, the proposed scheme is designed to follow the masking effect and visibility threshold more closely, i.e., the case when both masked and masking signals are small is more effectively tackled by the proposed scheme. Finally, the effects of the changes in luminance and contrast-structure are integrated via an adaptive method to obtain the overall image quality score. Extensive experiments conducted with six publicly available subject-rated databases (comprising of diverse images and distortion types) have confirmed the effectiveness, robustness, and efficiency of the proposed scheme in comparison with the relevant state-of-the-art schemes.

Monotonic regression: a new way for correlating subjective and objective ratings in image quality research

To assess the performance of image quality metrics (IQMs), some regressions, such as logistic regression and polynomial regression, are used to correlate objective ratings with subjective scores. However, some defects in optimality are shown in these regressions. In this correspondence, monotonic regression (MR) is found to be an effective correlation method in the performance assessment of IQMs. Both theoretical analysis and experimental results have proven that MR performs better than any other regression. We believe that MR could be an effective tool for performance assessment in the IQM research.

Quaternion structural similarity: a new quality index for color images

One of the most important issues for researchers developing image processing algorithms is image quality. Methodical quality evaluation, by showing images to several human observers, is slow, expensive, and highly subjective. On the other hand, a visual quality matrix (VQM) is a fast, cheap, and objective tool for evaluating image quality. Although most VQMs are good in predicting the quality of an image degraded by a single degradation, they poorly perform for a combination of two degradations. An example for such degradation is the color crosstalk (CTK) effect, which introduces blur with desaturation. CTK is expected to become a bigger issue in image quality as the industry moves toward smaller sensors. In this paper, we will develop a VQM that will be able to better evaluate the quality of an image degraded by a combined blur/desaturation degradation and perform as well as other VQMs on single degradations such as blur, compression, and noise. We show why standard scalar techniques are insufficient to measure a combined blur/desaturation degradation and explain why a vectorial approach is better suited. We introduce quaternion image processing (QIP), which is a true vectorial approach and has many uses in the fields of physics and engineering. Our new VQM is a vectorial expansion of structure similarity using QIP, which gave it its name-Quaternion Structural SIMilarity (QSSIM). We built a new database of a combined blur/desaturation degradation and conducted a quality survey with human subjects. An extensive comparison between QSSIM and other VQMs on several image quality databases-including our new database-shows the superiority of this new approach in predicting visual quality of color images.

Image quality assessment by visual gradient similarity

A full-reference image quality assessment (IQA) model by multiscale visual gradient similarity (VGS) is presented. The VGS model adopts a three-stage approach: First, global contrast registration for each scale is applied. Then, pointwise comparison is given by multiplying the similarity of gradient direction with the similarity of gradient magnitude. Third, intrascale pooling is applied, followed by interscale pooling. Several properties of human visual systems on image gradient have been explored and incorporated into the VGS model. It has been found that Stevens' power law is also suitable for gradient magnitude. Other factors such as quality uniformity, visual detection threshold of gradient, and visual frequency sensitivity also affect subjective image quality. The optimal values of two parameters of VGS are trained with existing IQA databases, and good performance of VGS has been verified by cross validation. Experimental results show that VGS is competitive with state-of-the-art metrics in terms of prediction precision, reliability, simplicity, and low computational cost.

SVD-based quality metric for image and video using machine learning

We study the use of machine learning for visual quality evaluation with comprehensive singular value decomposition (SVD)-based visual features. In this paper, the two-stage process and the relevant work in the existing visual quality metrics are first introduced followed by an in-depth analysis of SVD for visual quality assessment. Singular values and vectors form the selected features for visual quality assessment. Machine learning is then used for the feature pooling process and demonstrated to be effective. This is to address the limitations of the existing pooling techniques, like simple summation, averaging, Minkowski summation, etc., which tend to be ad hoc. We advocate machine learning for feature pooling because it is more systematic and data driven. The experiments show that the proposed method outperforms the eight existing relevant schemes. Extensive analysis and cross validation are performed with ten publicly available databases (eight for images with a total of 4042 test images and two for video with a total of 228 videos). We use all publicly accessible software and databases in this study, as well as making our own software public, to facilitate comparison in future research.

Objective image quality assessment based on support vector regression

Objective image quality estimation is useful in many visual processing systems, and is difficult to perform in line with the human perception. The challenge lies in formulating effective features and fusing them into a single number to predict the quality score. In this brief, we propose a new approach to address the problem, with the use of singular vectors out of singular value decomposition (SVD) as features for quantifying major structural information in images and then support vector regression (SVR) for automatic prediction of image quality. The feature selection with singular vectors is novel and general for gauging structural changes in images as a good representative of visual quality variations. The use of SVR exploits the advantages of machine learning with the ability to learn complex data patterns for an effective and generalized mapping of features into a desired score, in contrast with the oft-utilized feature pooling process in the existing image quality estimators; this is to overcome the difficulty of model parameter determination for such a system to emulate the related, complex human visual system (HVS) characteristics. Experiments conducted with three independent databases confirm the effectiveness of the proposed system in predicting image quality with better alignment with the HVS's perception than the relevant existing work. The tests with untrained distortions and databases further demonstrate the robustness of the system and the importance of the feature selection.

VSNR: a wavelet-based visual signal-to-noise ratio for natural images

This paper presents an efficient metric for quantifying the visual fidelity of natural images based on near-threshold and suprathreshold properties of human vision. The proposed metric, the visual signal-to-noise ratio (VSNR), operates via a two-stage approach. In the first stage, contrast thresholds for detection of distortions in the presence of natural images are computed via wavelet-based models of visual masking and visual summation in order to determine whether the distortions in the distorted image are visible. If the distortions are below the threshold of detection, the distorted image is deemed to be of perfect visual fidelity (VSNR = infinity) and no further analysis is required. If the distortions are suprathreshold, a second stage is applied which operates based on the low-level visual property of perceived contrast, and the mid-level visual property of global precedence. These two properties are modeled as Euclidean distances in distortion-contrast space of a multiscale wavelet decomposition, and VSNR is computed based on a simple linear sum of these distances. The proposed VSNR metric is generally competitive with current metrics of visual fidelity; it is efficient both in terms of its low computational complexity and in terms of its low memory requirements; and it operates based on physical luminances and visual angle (rather than on digital pixel values and pixel-based dimensions) to accommodate different viewing conditions.