Fast and Efficient Image Novelty Detection Based on Mean-Shifts

Image novelty detection is a repeating task in computer vision and describes the detection of anomalous images based on a training dataset consisting solely of normal reference data. It has been found that, in particular, neural networks are well-suited for the task. Our approach first transforms the training and test images into ensembles of patches, which enables the assessment of mean-shifts between normal data and outliers. As mean-shifts are only detectable when the outlier ensemble and inlier distribution are spatially separate from each other, a rich feature space, such as a pre-trained neural network, needs to be chosen to represent the extracted patches. For mean-shift estimation, the Hotelling T2 test is used. The size of the patches turned out to be a crucial hyperparameter that needs additional domain knowledge about the spatial size of the expected anomalies (local vs. global). This also affects model selection and the chosen feature space, as commonly used Convolutional Neural Networks or Vision Image Transformers have very different receptive field sizes. To showcase the state-of-the-art capabilities of our approach, we compare results with classical and deep learning methods on the popular dataset CIFAR-10, and demonstrate its real-world applicability in a large-scale industrial inspection scenario using the MVTec dataset. Because of the inexpensive design, our method can be implemented by a single additional 2D-convolution and pooling layer and allows particularly fast prediction times while being very data-efficient.

Complex echo classification by echo-locating bats: a review

Echo-locating bats constantly emit ultrasonic pulses and analyze the returning echoes to detect, localize, and classify objects in their surroundings. Echo classification is essential for bats' everyday life; for instance, it enables bats to use acoustical landmarks for navigation and to recognize food sources from other objects. Most of the research of echo based object classification in echo-locating bats was done in the context of simple artificial objects. These objects might represent prey, flower, or fruit and are characterized by simple echoes with a single up to several reflectors. Bats, however, must also be able to use echoes that return from complex structures such as plants or other types of background. Such echoes are characterized by superpositions of many reflections that can only be described using a stochastic statistical approach. Scientists have only lately started to address the issue of complex echo classification by echo-locating bats. Some behavioral evidence showing that bats can classify complex echoes has been accumulated and several hypotheses have been suggested as to how they do so. Here, we present a first review of this data. We raise some hypotheses regarding possible interpretations of the data and point out necessary future directions that should be pursued.

What a plant sounds like: the statistics of vegetation echoes as received by echolocating bats

A critical step on the way to understanding a sensory system is the analysis of the input it receives. In this work we examine the statistics of natural complex echoes, focusing on vegetation echoes. Vegetation echoes constitute a major part of the sensory world of more than 800 species of echolocating bats and play an important role in several of their daily tasks. Our statistical analysis is based on a large collection of plant echoes acquired by a biomimetic sonar system. We explore the relation between the physical world (the structure of the plant) and the characteristics of its echo. Finally, we complete the story by analyzing the effect of the sensory processing of both the echolocation and the auditory systems on the echoes and interpret them in the light of information maximization. The echoes of all different plant species we examined share a surprisingly robust pattern that was also reproduced by a simple Poisson model of the spatial reflector arrangement. The fine differences observed between the echoes of different plant species can be explained by the spatial characteristics of the plants. The bat's emitted signal enhances the most informative spatial frequency range where the species-specific information is large. The auditory system filtering affects the echoes in a similar way, thus enhancing the most informative spatial frequency range even more. These findings suggest how the bat's sensory system could have evolved to deal with complex natural echoes.

The voice of bats: how greater mouse-eared bats recognize individuals based on their echolocation calls

Echolocating bats use the echoes from their echolocation calls to perceive their surroundings. The ability to use these continuously emitted calls, whose main function is not communication, for recognition of individual conspecifics might facilitate many of the social behaviours observed in bats. Several studies of individual-specific information in echolocation calls found some evidence for its existence but did not quantify or explain it. We used a direct paradigm to show that greater mouse-eared bats (Myotis myotis) can easily discriminate between individuals based on their echolocation calls and that they can generalize their knowledge to discriminate new individuals that they were not trained to recognize. We conclude that, despite their high variability, broadband bat-echolocation calls contain individual-specific information that is sufficient for recognition. An analysis of the call spectra showed that formant-related features are suitable cues for individual recognition. As a model for the bat's decision strategy, we trained nonlinear statistical classifiers to reproduce the behaviour of the bats, namely to repeat correct and incorrect decisions of the bats. The comparison of the bats with the model strongly implies that the bats are using a prototype classification approach: they learn the average call characteristics of individuals and use them as a reference for classification.

Animals must recognize each other in order to engage in social behaviour. Vocal communication signals could be helpful for recognizing individuals, especially in nocturnal organisms such as bats. Echolocating bats continuously emit special vocalizations, known as echolocation calls, and perceive their surroundings by analyzing the returning echoes. In this work we show that bats can use these vocalizations for the recognition of individuals, despite the fact that their main function is not communication. We used a statistical approach to analyze how the bats could do so. We created a computer model that reproduces the recognition behaviour of the bats. Our model suggests that the bats learn the average calls of other individuals and recognize individuals by comparing their calls with the learnt average representations.

Center-surround patterns emerge as optimal predictors for human saccade targets

The human visual system is foveated, that is, outside the central visual field resolution and acuity drop rapidly. Nonetheless much of a visual scene is perceived after only a few saccadic eye movements, suggesting an effective strategy for selecting saccade targets. It has been known for some time that local image structure at saccade targets influences the selection process. However, the question of what the most relevant visual features are is still under debate. Here we show that center-surround patterns emerge as the optimal solution for predicting saccade targets from their local image structure. The resulting model, a one-layer feed-forward network, is surprisingly simple compared to previously suggested models which assume much more complex computations such as multi-scale processing and multiple feature channels. Nevertheless, our model is equally predictive. Furthermore, our findings are consistent with neurophysiological hardware in the superior colliculus. Bottom-up visual saliency may thus not be computed cortically as has been thought previously.

A Unifying View of Wiener and Volterra Theory and Polynomial Kernel Regression

Volterra and Wiener series are perhaps the best-understood nonlinear system representations in signal processing. Although both approaches have enjoyed a certain popularity in the past, their application has been limited to rather low-dimensional and weakly nonlinear systems due to the exponential growth of the number of terms that have to be estimated. We show that Volterra and Wiener series can be represented implicitly as elements of a reproducing kernel Hilbert space by using polynomial kernels. The estimation complexity of the implicit representation is linear in the input dimensionality and independent of the degree of nonlinearity. Experiments show performance advantages in terms of convergence, interpretability, and system sizes that can be handled.

Iterative kernel principal component analysis for image modeling

In recent years, Kernel Principal Component Analysis (KPCA) has been suggested for various image processing tasks requiring an image model such as, e.g., denoising or compression. The original form of KPCA, however, can be only applied to strongly restricted image classes due to the limited number of training examples that can be processed. We therefore propose a new iterative method for performing KPCA, the Kernel Hebbian Algorithm which iteratively estimates the Kernel Principal Components with only linear order memory complexity. In our experiments, we compute models for complex image classes such as faces and natural images which require a large number of training examples. The resulting image models are tested in single-frame super-resolution and denoising applications. The KPCA model is not specifically tailored to these tasks; in fact, the same model can be used in super-resolution with variable input resolution, or denoising with unknown noise characteristics. In spite of this, both super-resolution and denoising performance are comparable to existing methods.

Insect-Inspired Estimation of Egomotion

Tangential neurons in the fly brain are sensitive to the typical optic flow patterns generated during egomotion. In this study, we examine whether a simplified linear model based on the organization principles in tangential neurons can be used to estimate egomotion from the optic flow. We present a theory for the construction of an estimator consisting of a linear combination of optic flow vectors that incorporates prior knowledge about the distance distribution of the environment and about the noise and egomotion statistics of the sensor. The estimator is tested on a gantry carrying an omnidirectional vision sensor. The experiments show that the proposed approach leads to accurate and robust estimates of rotation rates, whereas translation estimates are of reasonable quality, albeit less reliable.

Wide-field, motion-sensitive neurons and matched filters for optic flow fields

The receptive field organization of a class of visual interneurons in the fly brain (vertical system, or VS neurons) shows a striking similarity to certain self-motion-induced optic flow fields. The present study compares the measured motion sensitivities of the VS neurons (Krapp et al. 1998) to a matched filter model for optic flow fields generated by rotation or translation. The model minimizes the variance of the filter output caused by noise and distance variability between different scenes. To that end, prior knowledge about distance and self-motion statistics is incorporated in the form of a "world model". We show that a special case of the matched filter model is able to predict the local motion sensitivities observed in some VS neurons. This suggests that their receptive field organization enables the VS neurons to maintain a consistent output when the same type of self-motion occurs in different situations.