Image quality assessment: from error visibility to structural similarity

A full-reference quality metric for geometrically distorted images

In multimedia applications, there has been an increasing interest in the use of quality measures based on human perception; however, research has not dealt with distortions due to geometric transformations. In this paper, we propose a method to objectively assess the perceptual quality of geometrically distorted images, based on image features processed by human vision. The proposed approach is a full-reference image quality metric focusing on the problem of local geometric distortions and is based on the use of Gabor filters that have received considerable attention because the characteristics of certain cells in the visual cortex of some mammals can be approximated by these filters. The novelty of the proposed technique is that it considers both the displacement field describing the distortion and the structure of the image. The experimental results show the good performances of the proposed metric.

Divisive normalization image quality metric revisited

Structural similarity metrics and information-theory-based metrics have been proposed as completely different alternatives to the traditional metrics based on error visibility and human vision models. Three basic criticisms were raised against the traditional error visibility approach: (1) it is based on near-threshold performance, (2) its geometric meaning may be limited, and (3) stationary pooling strategies may not be statistically justified. These criticisms and the good performance of structural and information-theory-based metrics have popularized the idea of their superiority over the error visibility approach. In this work we experimentally or analytically show that the above criticisms do not apply to error visibility metrics that use a general enough divisive normalization masking model. Therefore, the traditional divisive normalization metric 1 is not intrinsically inferior to the newer approaches. In fact, experiments on a number of databases including a wide range of distortions show that divisive normalization is fairly competitive with the newer approaches, robust, and easy to interpret in linear terms. These results suggest that, despite the criticisms of the traditional error visibility approach, divisive normalization masking models should be considered in the image quality discussion.

Contourlet-based image watermarking using optimum detector in a noisy environment

In this paper, an improved multiplicative image watermarking system is presented. Since human visual system is less sensitive to the image edges, watermarking is applied in the contourlet domain, which represents image edges sparsely. In the presented scheme, watermark data is embedded in directional subband with the highest energy. By modeling the contourlet coefficients with General Gaussian Distribution (GGD), the distribution of watermarked noisy coefficients is analytically calculated. The tradeoff between the transparency and robustness of the watermark data is solved in a novel fashion. At the receiver, based on the Maximum Likelihood (ML) decision rule, an optimal detector by the aid of channel side information is proposed. In the next step, a blind extension of the suggested algorithm is presented using the patchwork idea. Experimental results confirm the superiority of the proposed method against common attacks, such as Additive White Gaussian Noise (AWGN), JPEG compression, and rotation attacks, in comparison with the recently proposed techniques.

Simulation of 3D objects into breast tomosynthesis images

Digital breast tomosynthesis is a new three-dimensional (3D) breast-imaging modality that produces images of cross-sectional planes parallel to the detector plane from a limited number of X-ray projections over a limited angular range. Several technical and clinical parameters have not yet been completely optimised. Some of the open questions could be addressed experimentally; other parameter settings cannot be easily realised in practice and the associated optimisation process requires therefore a theoretical approach. Rather than simulating the complete 3D imaging chain, it is hypothesised that the simulation of small lesions into clinical (or test object) images can be of help in the optimisation process. In the present study, small 3D objects have been simulated into real projection images. Subsequently, these hybrid projection images are reconstructed using the routine clinical reconstruction tools. In this study, the validation of this simulation framework is reported through the comparison between simulated and real objects in reconstructed planes. The results confirm that there is no statistically significant difference between the simulated and the real objects. This suggests that other small mathematical or physiological objects could be simulated with the same approach.

Automated compensation of light attenuation in confocal microscopy by exact histogram specification

Confocal laser scanning microscopy (CLSM) enables us to capture images representing optical sections on the volume of a specimen. The images acquired from different layers have a different contrast: the images obtained from the deeper layers of the specimen will have a lower contrast with respect to the images obtained from the topmost layers. The main reasons responsible for the effects described above are light absorption and scattering by the atoms and molecules contained in the volume through which the light passes. Also light attenuation can be caused by the inclination of the observed surface. In the case of the surfaces that have a steep inclination, the reflected light will have a different direction than the one of the detector. We propose a technique of digital image processing that can be used to compensate the effects of light attenuation based on histogram operations. We process the image series obtained by CLSM by exact histogram specification and equalization. In this case, a strict ordering among pixels must be induced in order to achieve the exact histogram modeling. The processed images will end up having exactly the specified histogram and not a histogram with a shape that just resembles to the specified one, as in the case of classical histogram specification algorithms. Experimental results and theoretical aspects of the induced ordering are discussed, as well as a comparison between several histogram modeling techniques with respect to the processing of image series obtained by confocal microscopy.

Encoding natural scenes with neural circuits with random thresholds

We present a general framework for the reconstruction of natural video scenes encoded with a population of spiking neural circuits with random thresholds. The natural scenes are modeled as space-time functions that belong to a space of trigonometric polynomials. The visual encoding system consists of a bank of filters, modeling the visual receptive fields, in cascade with a population of neural circuits, modeling encoding in the early visual system. The neuron models considered include integrate-and-fire neurons and ON-OFF neuron pairs with threshold-and-fire spiking mechanisms. All thresholds are assumed to be random. We demonstrate that neural spiking is akin to taking noisy measurements on the stimulus both for time-varying and space-time-varying stimuli. We formulate the reconstruction problem as the minimization of a suitable cost functional in a finite-dimensional vector space and provide an explicit algorithm for stimulus recovery. We also present a general solution using the theory of smoothing splines in Reproducing Kernel Hilbert Spaces. We provide examples of both synthetic video as well as for natural scenes and demonstrate that the quality of the reconstruction degrades gracefully as the threshold variability of the neurons increases.

A versatile model for packet loss visibility and its application to packet prioritization

In this paper, we propose a generalized linear model for video packet loss visibility that is applicable to different group-of-picture structures. We develop the model using three subjective experiment data sets that span various encoding standards (H.264 and MPEG-2), group-of-picture structures, and decoder error concealment choices. We consider factors not only within a packet, but also in its vicinity, to account for possible temporal and spatial masking effects. We discover that the factors of scene cuts, camera motion, and reference distance are highly significant to the packet loss visibility. We apply our visibility model to packet prioritization for a video stream; when the network gets congested at an intermediate router, the router is able to decide which packets to drop such that visual quality of the video is minimally impacted. To show the effectiveness of our visibility model and its corresponding packet prioritization method, experiments are done to compare our perceptual-quality-based packet prioritization approach with existing Drop-Tail and Hint-Track-inspired cumulative-MSE-based prioritization methods. The result shows that our prioritization method produces videos of higher perceptual quality for different network conditions and group-of-picture structures. Our model was developed using data from high encoding-rate videos, and designed for high-quality video transported over a mostly reliable network; however, the experiments show the model is applicable to different encoding rates.

Low-complexity transcoding of JPEG images with near-optimal quality using a predictive quality factor and scaling parameters

A common transcoding operation consists of reducing the file size of a JPEG image to meet bandwidth or device constraints. This can be achieved by reducing its quality factor (QF) or reducing its resolution, or both. In this paper, using the Structural SIMilarity (SSIM) index as the quality metric, we present a system capable of estimating the QF and scaling parameters to achieve optimal quality while meeting a device's constraints. We then propose a novel low-complexity JPEG transcoding system which delivers near-optimal quality. The system is capable of predicting the best combination of QF and scaling parameters for a wide range of device constraints and viewing conditions. Although its computational complexity is an order of magnitude smaller than the system providing optimal quality, the proposed system yields quality results very similar to those of the optimal system.

The kinin B(2) receptor gene structure, product processing and expression in adult and fetal rats: evidence for gene evolution

We examined the structure of the rat kinin B2 receptor gene (KB2r) and encoding messenger RNA (mRNA) processing. Differently from the closely related mouse and rabbit genes that have three exons and two introns, the rat gene purportedly consists of four exons and three introns. There are two purported gene products; one of them contains an upstream approximately 180-bp open reading frame region ("exon-X") potentially expressed as a result of alternative processing. To examine the processing of rat KB2r mRNA, cDNA amplicons were generated using primer pairs directed towards 5' or 3' exon or intron flanking regions. Analyses of intron/exon primary cDNA amplicons showed that introns 1 to 3 are removed sequentially and that "exon-X" removal follows that of intron-3. No evidence was found for "exon-X" expression in polyadenylated (mature) mRNA of adult Wistar, Wistar Kyoto, spontaneously hypertensive or Sprague-Dawley rat tissues. Nor was "exon-X" detected in tissues subject to inflammatory stimulus expressing B1 kinin receptor mRNA or in 1- to 21-day-old rat embryos or fetuses. The lack of evidence for the expression of "exon-X" in mature mRNA indicates that the structure of the rat gene is similar to that of the mouse, rabbit and human genes, all consisting of three exons and two introns. The "exon-X" fragment may result from interstitial gene duplication, be a fragment of the ancestral gene, or most likely heterologous transposon insertion of an exon-like fragment into intron-2 of the KB2r gene.

Accelerating volume rendering by ray leaping with back steps

The methods for visualizing sampled spatial scientific data are known as volume rendering, where images are generated by computing 2D projections of 3D volume data. Since all the discrete data cells participate in the generation of each image, rendering time grows linearly with the resolution and complexity of the dataset. Empty cells in the data, which do not contribute to the final image, are of the important factors that increase the rendering time. During recent years, researchers have highly concentrated on improving the performance of these methods to achieve real time rendering. Skipping the empty space provides significant speedup and known as space leaping which requires implementation of special data structures and pre-processing. This paper presents a simple and efficient technique, that we name "ray-leaping," for the acceleration of total rendering process and eliminates the need for special data structures and pre-processing.

Motion tuned spatio-temporal quality assessment of natural videos

There has recently been a great deal of interest in the development of algorithms that objectively measure the integrity of video signals. Since video signals are being delivered to human end users in an increasingly wide array of applications and products, it is important that automatic methods of video quality assessment (VQA) be available that can assist in controlling the quality of video being delivered to this critical audience. Naturally, the quality of motion representation in videos plays an important role in the perception of video quality, yet existing VQA algorithms make little direct use of motion information, thus limiting their effectiveness. We seek to ameliorate this by developing a general, spatio-spectrally localized multiscale framework for evaluating dynamic video fidelity that integrates both spatial and temporal (and spatio-temporal) aspects of distortion assessment. Video quality is evaluated not only in space and time, but also in space-time, by evaluating motion quality along computed motion trajectories. Using this framework, we develop a full reference VQA algorithm for which we coin the term the MOtion-based Video Integrity Evaluation index, or MOVIE index. It is found that the MOVIE index delivers VQA scores that correlate quite closely with human subjective judgment, using the Video Quality Expert Group (VQEG) FRTV Phase 1 database as a test bed. Indeed, the MOVIE index is found to be quite competitive with, and even outperform, algorithms developed and submitted to the VQEG FRTV Phase 1 study, as well as more recent VQA algorithms tested on this database.

Objective image quality assessment based on support vector regression

Objective image quality estimation is useful in many visual processing systems, and is difficult to perform in line with the human perception. The challenge lies in formulating effective features and fusing them into a single number to predict the quality score. In this brief, we propose a new approach to address the problem, with the use of singular vectors out of singular value decomposition (SVD) as features for quantifying major structural information in images and then support vector regression (SVR) for automatic prediction of image quality. The feature selection with singular vectors is novel and general for gauging structural changes in images as a good representative of visual quality variations. The use of SVR exploits the advantages of machine learning with the ability to learn complex data patterns for an effective and generalized mapping of features into a desired score, in contrast with the oft-utilized feature pooling process in the existing image quality estimators; this is to overcome the difficulty of model parameter determination for such a system to emulate the related, complex human visual system (HVS) characteristics. Experiments conducted with three independent databases confirm the effectiveness of the proposed system in predicting image quality with better alignment with the HVS's perception than the relevant existing work. The tests with untrained distortions and databases further demonstrate the robustness of the system and the importance of the feature selection.

Impact of JPEG 2000 compression on lesion detection in MR imaging

PURPOSE

Lossy compression algorithms enable the efficient transmission of large medical image datasets over bandwidth-limited digital networks or facilitate the long-term storage of the daily image production. Although JPEG 2000 has been adopted by DICOM as the standard for lossy-to-lossless compression, still a set of guidelines needs to be derived that allows for the usage of the lossy mode, potentially jeopardizing the accuracy of lesion detection and characterization, and so, of the resulting diagnosis task effectiveness in the medical diagnosis process. In this article the authors present and evaluate a generic methodology for the determination of the minimal bit rate that still ensures an accurate detection in magnetic resonance images of specific lesions. In this article, they demonstrate the methodology for two particular pathologies, i.e., multiple sclerosis and Virchow-Robin space enlargements.

METHODS

Involving qualified personnel, the minimal bit rate is estimated from ROC experiments initially simulated through mathematical observers that are designed with several objective metrics. The mathematical observer models included three variants of the Hotelling observer plus the non-prewhitening matched filter with eye model, while the objective measures are based on distance, correlation, singular value decomposition, and structural similarity.

RESULTS

The results indicate that the highest compression without seriously affecting the detection of the studied lesions is achieved at a bit rate of 0.125 bpp. At this value, the detection effectiveness exceeded 95% with less than 5% standard deviation, while only 4.4% of the outcomes were classified as false negatives by the experts and 11.6% as false positives.

CONCLUSIONS

The optimal bit rate found assures that important information on the small investigated structures is still preserved for their accurate detection and their a posteriori characterization.

Provisioning of medical quality of services for HSDPA and mobile WiMAX in healthcare applications

Mobile healthcare, or m-health, is an evolutionary concept that provides both mobility and an 'always connected' healthcare functionality. The development of this concept depends on how best the available bandwidth in (HSDPA/HSUPA) and emerging (Mobile WiMAX) networks can be correlated with the relevant medical quality of services issues. In this paper we address and discuss some of these issues and challenges. We also provide an example of a bandwidth demanding application to verify such provision mechanisms.

Vector lifting schemes for stereo image coding

Many research efforts have been devoted to the improvement of stereo image coding techniques for storage or transmission. In this paper, we are mainly interested in lossy-to-lossless coding schemes for stereo images allowing progressive reconstruction. The most commonly used approaches for stereo compression are based on disparity compensation techniques. The basic principle involved in this technique first consists of estimating the disparity map. Then, one image is considered as a reference and the other is predicted in order to generate a residual image. In this paper, we propose a novel approach, based on vector lifting schemes (VLS), which offers the advantage of generating two compact multiresolution representations of the left and the right views. We present two versions of this new scheme. A theoretical analysis of the performance of the considered VLS is also conducted. Experimental results indicate a significant improvement using the proposed structures compared with conventional methods.

Training an active random field for real-time image denoising

Many computer vision problems can be formulated in a Bayesian framework based on Markov random fields (MRF) or conditional random fields (CRF). Generally, the MRF/CRF model is learned independently of the inference algorithm that is used to obtain the final result. In this paper, we observe considerable gains in speed and accuracy by training the MRF/CRF model together with a fast and suboptimal inference algorithm. An active random field (ARF) is defined as a combination of a MRF/CRF based model and a fast inference algorithm for the MRF/CRF model. This combination is trained through an optimization of a loss function and a training set consisting of pairs of input images and desired outputs. We apply the ARF concept to image denoising, using the Fields of Experts MRF together with a 1-4 iteration gradient descent algorithm for inference. Experimental validation on unseen data shows that the ARF approach obtains an improved benchmark performance as well as a 1000-3000 times speedup compared to the Fields of Experts MRF. Using the ARF approach, image denoising can be performed in real-time, at 8fps on a single CPU for a 256 x 256 image sequence, with close to state-of-the-art accuracy.

Complex wavelet structural similarity: a new image similarity index

We introduce a new measure of image similarity called the complex wavelet structural similarity (CW-SSIM) index and show its applicability as a general purpose image similarity index. The key idea behind CW-SSIM is that certain image distortions lead to consistent phase changes in the local wavelet coefficients, and that a consistent phase shift of the coefficients does not change the structural content of the image. By conducting four case studies, we have demonstrated the superiority of the CW-SSIM index against other indices (e.g., Dice, Hausdorff distance) commonly used for assessing the similarity of a given pair of images. In addition, we show that the CW-SSIM index has a number of advantages. It is robust to small rotations and translations. It provides useful comparisons even without a preprocessing image registration step, which is essential for other indices. Moreover, it is computationally less expensive.

Noise-driven anisotropic diffusion filtering of MRI

A new filtering method to remove Rician noise from magnetic resonance images is presented. This filter relies on a robust estimation of the standard deviation of the noise and combines local linear minimum mean square error filters and partial differential equations for MRI, as the speckle reducing anisotropic diffusion did for ultrasound images. The parameters of the filter are automatically chosen from the estimated noise. This property improves the convergence rate of the diffusion while preserving contours, leading to more robust and intuitive filtering. The partial derivative equation of the filter is extended to a new matrix diffusion filter which allows a coherent diffusion based on the local structure of the image and on the corresponding oriented local standard deviations. This new filter combines volumetric, planar, and linear components of the local image structure. The numerical scheme is explained and visual and quantitative results on simulated and real data sets are presented. In the experiments, the new filter leads to the best results.

On the role of exponential splines in image interpolation

A Sobolev reproducing-kernel Hilbert space approach to image interpolation is introduced. The underlying kernels are exponential functions and are related to stochastic autoregressive image modeling. The corresponding image interpolants can be implemented effectively using compactly-supported exponential B-splines. A tight l(2) upper-bound on the interpolation error is then derived, suggesting that the proposed exponential functions are optimal in this regard. Experimental results indicate that the proposed interpolation approach with properly-tuned, signal-dependent weights outperforms currently available polynomial B-spline models of comparable order. Furthermore, a unified approach to image interpolation by ideal and nonideal sampling procedures is derived, suggesting that the proposed exponential kernels may have a significant role in image modeling as well. Our conclusion is that the proposed Sobolev-based approach could be instrumental and a preferred alternative in many interpolation tasks.

Optimization of block size for DCT-based medical image compression

In view of the increasing importance of medical imaging in healthcare and the large amount of image data to be transmitted/stored, the need for development of an efficient medical image compression method, which would preserve the critical diagnostic information at higher compression, is growing. Discrete cosine transform (DCT) is a popular transform used in many practical image/video compression systems because of its high compression performance and good computational efficiency. As the computational burden of full frame DCT would be heavy, the image is usually divided into non-overlapping sub-images, or blocks, for processing. This paper aims to identify the optimum size of the block, in reference to compression of CT, ultrasound and X-ray images. Three conflicting requirements are considered, namely processing time, compression ratio and the quality of the reconstructed image. The quantitative comparison of various block sizes has been carried out on the basis of benefit-to-cost ratio (BCR) and reconstruction quality score (RQS). Experimental results are presented that verify the optimality of the 16 x 16 block size.

Speckle reduction in ultrasound medical images using adaptive filter based on second order statistics

This article discusses an adaptive filtering technique for reducing speckle using second order statistics of the speckle pattern in ultrasound medical images. Several region-based adaptive filter techniques have been developed for speckle noise suppression, but there are no specific criteria for selecting the region growing size in the post processing of the filter. The size appropriate for one local region may not be appropriate for other regions. Selection of the correct region size involves a trade-off between speckle reduction and edge preservation. Generally, a large region size is used to smooth speckle and a small size to preserve the edges into an image. In this paper, a smoothing procedure combines the first order statistics of speckle for the homogeneity test and second order statistics for selection of filters and desired region growth. Grey level co-occurrence matrix (GLCM) is calculated for every region during the region contraction and region growing for second order statistics. Further, these GLCM features determine the appropriate filter for the region smoothing. The performance of this approach is compared with the aggressive region-growing filter (ARGF) using edge preservation and speckle reduction tests. The processed image results show that the proposed method effectively reduces speckle noise and preserves edge details.

Virtual slide telepathology workstation of the future: lessons learned from teleradiology

The clinical reading environment for the 21st century pathologist looks very different than it did even a few short years ago. Glass slides are quickly being replaced by digital "virtual slides," and the traditional light microscope is being replaced by the computer display. There are numerous questions that arise however when deciding exactly what this new digital display viewing environment will be like. Choosing a workstation for daily use in the interpretation of digital pathology images can be a very daunting task. Radiology went digital nearly 20 years ago and faced many of the same challenges so there are lessons to be learned from these experiences. One major lesson is that there is no "one size fits all" workstation so users must consider a variety of factors when choosing a workstation. In this article, we summarize some of the potentially critical elements in a pathology workstation and the characteristics one should be aware of and look for in the selection of one. Issues pertaining to both hardware and software aspects of medical workstations will be reviewed particularly as they may impact the interpretation process.

Subjective and objective quality assessment in wireless teleultrasonography imaging

Mobile Robotic teleultrasonography is an emerging technology that can be applied in different clinical settings for remote ultrasound scanning without the need of the expert at the point of care. Guaranteed medical image quality for diagnostic purposes and their delivery in bandwidth limited wireless environments is a challenging issue. In this paper we present some of the subjective and objective image analysis acquired from a robotic teleultrasonography system operated remotely by the expert to provide an assessment of these medical imaging measures for such advanced wireless telemedical system.

The effectiveness of detector combinations

In this paper, the performance improvement benefiting from the combination of local feature detectors for image matching and registration is evaluated. Possible combinations of five types of representative interest point detectors and region detectors are integrated into the testing framework. The performance is compared using the number of correspondences and the repeatability rate, as well as an original evaluation criterion named the Reconstruction Similarity (RS), which reflects not only the number of matches, but also the degree of matching error. It is observed that the combination of DoG extremum and MSCR outperforms any single detectors and other detector combinations in most cases. Furthermore, MDSS, a hybrid algorithm for accurate image matching, is proposed. Compared with standard SIFT and GLOH, its average RS rate exceeds more than 3.56%, and takes up even less computational time.

Correlation-based motion vector processing with adaptive interpolation scheme for motion-compensated frame interpolation

In this paper, we address the problems of unreliable motion vectors that cause visual artifacts but cannot be detected by high residual energy or bidirectional prediction difference in motion-compensated frame interpolation. A correlation-based motion vector processing method is proposed to detect and correct those unreliable motion vectors by explicitly considering motion vector correlation in the motion vector reliability classification, motion vector correction, and frame interpolation stages. Since our method gradually corrects unreliable motion vectors based on their reliability, we can effectively discover the areas where no motion is reliable to be used, such as occlusions and deformed structures. We also propose an adaptive frame interpolation scheme for the occlusion areas based on the analysis of their surrounding motion distribution. As a result, the interpolated frames using the proposed scheme have clearer structure edges and ghost artifacts are also greatly reduced. Experimental results show that our interpolated results have better visual quality than other methods. In addition, the proposed scheme is robust even for those video sequences that contain multiple and fast motions.

Noise Reduction for CFA Image Sensors Exploiting HVS Behaviour

This paper presents a spatial noise reduction technique designed to work on CFA (Color Filtering Array) data acquired by CCD/CMOS image sensors. The overall processing preserves image details using some heuristics related to the HVS (Human Visual System); estimates of local texture degree and noise levels are computed to regulate the filter smoothing capability. Experimental results confirm the effectiveness of the proposed technique. The method is also suitable for implementation in low power mobile devices with imaging capabilities such as camera phones and PDAs.

Subjective evaluation of spatial resolution and quantization noise tradeoffs

Most full-reference fidelity/quality metrics compare the original image to a distorted image at the same resolution assuming a fixed viewing condition. However, in many applications, such as video streaming, due to the diversity of channel capacities and display devices, the viewing distance and the spatiotemporal resolution of the displayed signal may be adapted in order to optimize the perceived signal quality. For example, at low bitrate coding applications an observer may prefer to reduce the resolution or increase the viewing distance to reduce the visibility of the compression artifacts. The tradeoff between resolution/viewing conditions and visibility of compression artifacts requires new approaches for the evaluation of image quality that account for both image distortions and image size. In order to better understand such tradeoffs, we conducted subjective tests using two representative still image coders, JPEG and JPEG 2000. Our results indicate that an observer would indeed prefer a lower spatial resolution (at a fixed viewing distance) in order to reduce the visibility of the compression artifacts, but not all the way to the point where the artifacts are completely invisible. Moreover, the observer is willing to accept more artifacts as the image size decreases. The subjective test results we report can be used to select viewing conditions for coding applications. They also set the stage for the development of novel fidelity metrics. The focus of this paper is on still images, but it is expected that similar tradeoffs apply to video.