Statistical-mechanical approach to image processing

Ghost imaging with Bayesian denoising method

We propose a Bayesian denoising method to improve the quality of ghost imaging. The proposed method achieved the highest PSNR and SSIM in both binary and gray-scale targets with fewer measurements. Experimentally, it obtained a reconstructed image of a USAF target where the PSNR and SSIM of the image were up to 12.80 dB and 0.77, respectively, whereas those of traditional ghost images were 7.24 dB and 0.28 with 3000 measurements. Furthermore, it was robust against additive Gaussian noise. Thus, this method could make the ghost imaging technique more feasible as a practical application.

Statistical mechanical evaluation of a spread-spectrum watermarking model with image restoration

In cases in which an original image is blind, a decoding method where both the image and the messages can be estimated simultaneously is desirable. We propose a spread spectrum watermarking model with image restoration based on Bayes estimation. We therefore need to assume some prior probabilities. The probability for estimating the messages is given by the uniform distribution, and the ones for the image are given by the infinite-range model and two-dimensional (2D) Ising model. Any attacks from unauthorized users can be represented by channel models. We can obtain the estimated messages and image by maximizing the posterior probability. We analyzed the performance of the proposed method by the replica method in the case of the infinite-range model. We first calculated the theoretical values of the bit error rate from obtained saddle-point equations and then verified them by computer simulations. For this purpose, we assumed that the image is binary and is generated from a given prior probability. We also assume that attacks can be represented by the Gaussian channel. The computer simulation retults agreed with the theoretical values. In the case of prior probability given by the 2D Ising model, in which each pixel is statically connected with four-neighbors, we evaluated the decoding performance by computer simulations, since the replica theory could not be applied. Results using the 2D Ising model showed that the proposed method with image restoration is as effective as the infinite-range model for decoding messages. We compared the performances in a case in which the image was blind and one in which it was informed. The difference between these cases was small as long as the embedding and attack rates were small. This demonstrates that the proposed method with simultaneous estimation is effective as a watermarking decoder.

A Mechanical Approach for Smooth Surface Fitting to Delineate Vessel Walls in Optical Coherence Tomography Images

Automated analysis of vascular imaging techniques is limited by the inability to precisely determine arterial borders. Intravascular optical coherence tomography (OCT) offers unprecedented detail of artery wall structure and composition, but does not provide consistent visibility of the outer border of the vessel due to the limited penetration depth. Existing interpolation and surface fitting methods prove insufficient to accurately fill the gaps between the irregularly spaced and sometimes unreliably identified visible segments of the vessel outer border. This paper describes an intuitive, efficient, and flexible new method of 3D surface fitting and smoothing suitable for this task. An anisotropic linear-elastic mesh is fit to irregularly spaced and uncertain data points corresponding to visible segments of vessel borders, enabling the fully automated delineation of the entire inner and outer borders of diseased vessels in OCT images for the first time. In a clinical dataset, the proposed smooth surface fitting approach had great agreement when compared with human annotations: areas differed by just 11 ± 11% (0.93 ± 0.84 mm²), with a coefficient of determination of 0.89. Overlapping and non-overlapping area ratios were 0.91 and 0.18, respectively, with a sensitivity of 90.8 and specificity of 99.0. This spring mesh method of contour fitting significantly outperformed all alternative surface fitting and interpolation approaches tested. The application of this promising proposed method is expected to enhance clinical intervention and translational research using OCT.

Bayesian inversion analysis of nonlinear dynamics in surface heterogeneous reactions

It is essential to extract nonlinear dynamics from time-series data as an inverse problem in natural sciences. We propose a Bayesian statistical framework for extracting nonlinear dynamics of surface heterogeneous reactions from sparse and noisy observable data. Surface heterogeneous reactions are chemical reactions with conjugation of multiple phases, and they have the intrinsic nonlinearity of their dynamics caused by the effect of surface-area between different phases. We adapt a belief propagation method and an expectation-maximization (EM) algorithm to partial observation problem, in order to simultaneously estimate the time course of hidden variables and the kinetic parameters underlying dynamics. The proposed belief propagation method is performed by using sequential Monte Carlo algorithm in order to estimate nonlinear dynamical system. Using our proposed method, we show that the rate constants of dissolution and precipitation reactions, which are typical examples of surface heterogeneous reactions, as well as the temporal changes of solid reactants and products, were successfully estimated only from the observable temporal changes in the concentration of the dissolved intermediate product.

Statistical analysis of loopy belief propagation in random fields

Loopy belief propagation (LBP), which is equivalent to the Bethe approximation in statistical mechanics, is a message-passing-type inference method that is widely used to analyze systems based on Markov random fields (MRFs). In this paper, we propose a message-passing-type method to analytically evaluate the quenched average of LBP in random fields by using the replica cluster variation method. The proposed analytical method is applicable to general pairwise MRFs with random fields whose distributions differ from each other and can give the quenched averages of the Bethe free energies over random fields, which are consistent with numerical results. The order of its computational cost is equivalent to that of standard LBP. In the latter part of this paper, we describe the application of the proposed method to Bayesian image restoration, in which we observed that our theoretical results are in good agreement with the numerical results for natural images.

Markov-random-field modeling for linear seismic tomography

We apply the Markov-random-field model to linear seismic tomography and propose a method to estimate the hyperparameters for the smoothness and the magnitude of the noise. Optimal hyperparameters can be determined analytically by minimizing the free energy function, which is defined by marginalizing the evaluation function. In synthetic inversion tests under various settings, the assumed velocity structures are successfully reconstructed, which shows the effectiveness and robustness of the proposed method. The proposed mathematical framework can be applied to inversion problems in various fields in the natural sciences.

Defining statistical perceptions with an empirical Bayesian approach

Extracting statistical structures (including textures or contrasts) from a natural stimulus is a central challenge in both biological and engineering contexts. This study interprets the process of statistical recognition in terms of hyperparameter estimations and free-energy minimization procedures with an empirical Bayesian approach. This mathematical interpretation resulted in a framework for relating physiological insights in animal sensory systems to the functional properties of recognizing stimulus statistics. We applied the present theoretical framework to two typical models of natural images that are encoded by a population of simulated retinal neurons, and demonstrated that the resulting cognitive performances could be quantified with the Fisher information measure. The current enterprise yielded predictions about the properties of human texture perception, suggesting that the perceptual resolution of image statistics depends on visual field angles, internal noise, and neuronal information processing pathways, such as the magnocellular, parvocellular, and koniocellular systems. Furthermore, the two conceptually similar natural-image models were found to yield qualitatively different predictions, striking a note of warning against confusing the two models when describing a natural image.

Effect of quantum fluctuation in error-correcting codes

We discuss the decoding performance of error-correcting codes based on a model in which quantum fluctuations are introduced by means of a transverse field. The essential issue in this paper is whether quantum fluctuations improve the decoding quality compared with the conventional estimation based on thermal fluctuations, which is called finite-temperature decoding. We found that an estimation incorporating quantum fluctuations approaches the optimal performance of finite-temperature decoding. The results are illustrated by numerically solving saddle-point equations and performing a Monte Carlo simulation. We also evaluated the upper bound of the overlap between the original sequence and the decoded sequence derived from the equations of state for the order parameters, which is a measure of the decoding performance.

Replica inference approach to unsupervised multiscale image segmentation

We apply a replica-inference-based Potts model method to unsupervised image segmentation on multiple scales. This approach was inspired by the statistical mechanics problem of "community detection" and its phase diagram. Specifically, the problem is cast as identifying tightly bound clusters ("communities" or "solutes") against a background or "solvent." Within our multiresolution approach, we compute information-theory-based correlations among multiple solutions ("replicas") of the same graph over a range of resolutions. Significant multiresolution structures are identified by replica correlations manifest by information theory overlaps. We further employ such information theory measures (such as normalized mutual information and variation of information), thermodynamic quantities such as the system entropy and energy, and dynamic measures monitoring the convergence time to viable solutions as metrics for transitions between various solvable and unsolvable phases. Within the solvable phase, transitions between contending solutions (such as those corresponding to segmentations on different scales) may also appear. With the aid of these correlations as well as thermodynamic measures, the phase diagram of the corresponding Potts model is analyzed at both zero and finite temperatures. Optimal parameters corresponding to a sensible unsupervised segmentations appear within the "easy phase" of the Potts model. Our algorithm is fast and shown to be at least as accurate as the best algorithms to date and to be especially suited to the detection of camouflaged images.

Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications

In this paper we extend our previous work on the stochastic block model, a commonly used generative model for social and biological networks, and the problem of inferring functional groups or communities from the topology of the network. We use the cavity method of statistical physics to obtain an asymptotically exact analysis of the phase diagram. We describe in detail properties of the detectability-undetectability phase transition and the easy-hard phase transition for the community detection problem. Our analysis translates naturally into a belief propagation algorithm for inferring the group memberships of the nodes in an optimal way, i.e., that maximizes the overlap with the underlying group memberships, and learning the underlying parameters of the block model. Finally, we apply the algorithm to two examples of real-world networks and discuss its performance.

A bayesian hyperparameter inference for radon-transformed image reconstruction

We develop a hyperparameter inference method for image reconstruction from Radon transform which often appears in the computed tomography, in the manner of Bayesian inference. Hyperparameters are often introduced in Bayesian inference to control the strength ratio between prior information and the fidelity to the observation. Since the quality of the reconstructed image is controlled by the estimation accuracy of these hyperparameters, we apply Bayesian inference into the filtered back-projection (FBP) reconstruction method with hyperparameters inference and demonstrate that the estimated hyperparameters can adapt to the noise level in the observation automatically. In the computer simulation, at first, we show that our algorithm works well in the model framework environment, that is, observation noise is an additive white Gaussian noise case. Then, we also show that our algorithm works well in the more realistic environment, that is, observation noise is Poissonian noise case. After that, we demonstrate an application for the real chest CT image reconstruction under the Gaussian and Poissonian observation noises.

Inference in alpha rhythm phase and amplitude modeled on Markov random field using belief propagation from electroencephalograms

Alpha rhythm is a major component of spontaneous electroencephalographic (EEG) data. We develop a novel method that can be used to estimate the instantaneous phases and amplitudes of the alpha rhythm with high accuracy by modeling the alpha rhythm phase and amplitude as Markov random field (MRF) models. By using a belief propagation technique, we construct an exact-inference algorithm that can be used to estimate instantaneous phases and amplitudes and calculate the marginal likelihood. Maximizing the marginal likelihood enables us to estimate the hyperparameters on the basis of type-II maximum likelihood estimation. We prove that the instantaneous phase and amplitude estimation by our method is consistent with that by the Hilbert transform, which has been commonly used to estimate instantaneous phases and amplitudes, of a signal filtered from observed data in the limited case that the observed data consist of only one frequency signal whose amplitude is constant and a Gaussian noise. Comparison of the performances of observation noise reduction by our method and by a Gaussian MRF model of alpha rhythm signal indicates that our method reduces observation noise more efficiently. Moreover, the instantaneous phase and amplitude estimates obtained using our method are more accurate than those obtained by the Hilbert transform. Application of our method to experimental EEG data also demonstrates that the relationship between the alpha rhythm phase and the reaction time emerges more clearly by using our method than the Hilbert transform. This indicates our method's practical usefulness. Therefore, applying our method to experimental EEG data will enable us to estimate the instantaneous phases and amplitudes of the alpha rhythm more precisely.

Accuracy of Loopy belief propagation in Gaussian models

This paper considers the loopy belief propagation (LBP) algorithm applied to Gaussian graphical models. It is known for Gaussian belief propagation that, if LBP converges, LBP computes the exact posterior means but incorrect variances. In this paper, we analytically derive the posterior variances for some special structured graphs and clarify the accuracy of LBP. For the graphs of a single cycle, we derive a rigorous solution for the posterior variances and thereby find the quantity that determines the accuracy of LBP. Based on this result, we state a necessary condition for LBP convergence. The quantity above also plays an important role in graphs of a single cycle with arbitrary trees. For arbitrary topological graphs, we consider the situation where correlations between any pair of nodes are comparatively small and show analytically the principal values that determine the accuracy of LBP.

Superresolution with compound Markov random fields via the variational EM algorithm

This study deals with a reconstruction-type superresolution problem and the accompanying image registration problem simultaneously. We propose a Bayesian approach in which the prior is modeled as a compound Gaussian Markov random field (MRF) and marginalization is performed over unknown variables to avoid overfitting. Our algorithm not only avoids overfitting, but also preserves discontinuity in the estimated image, unlike existing single-layer Gaussian MRF models for Bayesian superresolution. Maximum-marginal-likelihood estimation of the registration parameters is carried out using a variational EM algorithm where hidden variables are marginalized out, and the posterior distribution is variationally approximated by a factorized trial distribution. High-resolution image estimates are obtained through the process of posterior computation in the EM algorithm. Experiments show that our Bayesian approach with the two-layer compound model exhibits better performance both in quantitative measures and visual quality than the single-layer model.

MAP estimation algorithm for phase response curves based on analysis of the observation process

Many research groups have sought to measure phase response curves (PRCs) from real neurons. However, methods of estimating PRCs from noisy spike-response data have yet to be established. In this paper, we propose a Bayesian approach for estimating PRCs. First, we analytically obtain a likelihood function of the PRC from a detailed model of the observation process formulated as Langevin equations. Then we construct a maximum a posteriori (MAP) estimation algorithm based on the analytically obtained likelihood function. The MAP estimation algorithm derived here is equivalent to the spherical spin model. Moreover, we analytically calculate a marginal likelihood corresponding to the free energy of the spherical spin model, which enables us to estimate the hyper-parameters, i.e., the intensity of the Langevin force and the smoothness of the prior.

Statistical-mechanical iterative algorithms on complex networks

The Ising models have been applied for various problems on information sciences, social sciences, and so on. In many cases, solving these problems corresponds to minimizing the Bethe free energy. To minimize the Bethe free energy, a statistical-mechanical iterative algorithm is often used. We study the statistical-mechanical iterative algorithm on complex networks. To investigate effects of heterogeneous structures on the iterative algorithm, we introduce an iterative algorithm based on information of heterogeneity of complex networks, in which higher-degree nodes are likely to be updated more frequently than lower-degree ones. Numerical experiments clarified that the usage of the information of heterogeneity affects the algorithm in Barabási and Albert networks, but does not influence that in Erdös and Rényi networks. It is revealed that information of the whole system propagates rapidly through such high-degree nodes in the case of Barabási-Albert's scale-free networks.