Two representations of a high-dimensional perceptual space

Perceptual thresholds for differences in CT noise texture

Purpose

The average (f av ) or peak (f peak ) noise power spectrum (NPS) frequency is often used as a one-parameter descriptor of the CT noise texture. Our study develops a more complete two-parameter model of the CT NPS and investigates the sensitivity of human observers to changes in it.

Approach

A model of CT NPS was created based on its f peak and a half-Gaussian fit (σ ) to the downslope. Two-alternative forced-choice staircase studies were used to determine perceptual thresholds for noise texture, defined as parameter differences with a predetermined level of discrimination performance (80% correct). Five imaging scientist observers performed the forced-choice studies for eight directions in the f peak / σ -space, for two reference NPSs (corresponding to body and lung kernels). The experiment was repeated with 32 radiologists, each evaluating a single direction in the f peak / σ -space. NPS differences were quantified by the noise texture contrast (C texture ), the integral of the absolute NPS difference.

Results

The two-parameter NPS model was found to be a good representation of various clinical CT reconstructions. Perception thresholds for f peak alone are 0.2    lp / cm for body and 0.4    lp / cm for lung NPSs. For σ , these values are 0.15 and 2    lp / cm , respectively. Thresholds change if the other parameter also changes. Different NPSs with the same f peak or f av can be discriminated. Nonradiologist observers did not need more C texture than radiologists.

Conclusions

f peak or f av is insufficient to describe noise texture completely. The discrimination of noise texture changes depending on its frequency content. Radiologists do not discriminate noise texture changes better than nonradiologists.

The Geometry of Low- and High-Level Perceptual Spaces

Low-level features are typically continuous (e.g., the gamut between two colors), but semantic information is often categorical (there is no corresponding gradient between dog and turtle) and hierarchical (animals live in land, water, or air). To determine the impact of these differences on cognitive representations, we characterized the geometry of perceptual spaces of five domains: a domain dominated by semantic information (animal names presented as words), a domain dominated by low-level features (colored textures), and three intermediate domains (animal images, lightly texturized animal images that were easy to recognize, and heavily texturized animal images that were difficult to recognize). Each domain had 37 stimuli derived from the same animal names. From 13 participants (9F), we gathered similarity judgments in each domain via an efficient psychophysical ranking paradigm. We then built geometric models of each domain for each participant, in which distances between stimuli accounted for participants' similarity judgments and intrinsic uncertainty. Remarkably, the five domains had similar global properties: each required 5-7 dimensions, and a modest amount of spherical curvature provided the best fit. However, the arrangement of the stimuli within these embeddings depended on the level of semantic information: dendrograms derived from semantic domains (word, image, and lightly texturized images) were more "tree-like" than those from feature-dominated domains (heavily texturized images and textures). Thus, the perceptual spaces of domains along this feature-dominated to semantic-dominated gradient shift to a tree-like organization when semantic information dominates, while retaining a similar global geometry.

Ordinal Characterization of Similarity Judgments

Characterizing judgments of similarity within a perceptual or semantic domain, and making inferences about the underlying structure of this domain from these judgments, has an increasingly important role in cognitive and systems neuroscience. We present a new framework for this purpose that makes very limited assumptions about how perceptual distances are converted into similarity judgments. The approach starts from a dataset of empirical judgments of relative similarities: the fraction of times that a subject chooses one of two comparison stimuli to be more similar to a reference stimulus. These empirical judgments provide Bayesian estimates of underling choice probabilities. From these estimates, we derive three indices that characterize the set of judgments, measuring consistency with a symmetric dis-similarity, consistency with an ultrametric space, and consistency with an additive tree. We illustrate this approach with example psychophysical datasets of dis-similarity judgments in several visual domains and provide code that implements the analyses.

Discrimination of textures with spatial correlations and multiple gray levels

Analysis of visual texture is important for many key steps in early vision. We study visual sensitivity to image statistics in three families of textures that include multiple gray levels and correlations in two spatial dimensions. Sensitivities to positive and negative correlations are approximately independent of correlation sign, and signals from different kinds of correlations combine quadratically. We build a computational model, fully constrained by prior studies of sensitivity to uncorrelated textures and black-and-white textures with spatial correlations. The model accounts for many features of the new data, including sign-independence, quadratic combination, and the dependence on gray-level distribution.

Functional recursion of orientation cues in figure-ground separation

Visual texture is an important cue to figure-ground organization. While processing of texture differences is a prerequisite for the use of this cue to extract figure-ground organization, these stages are distinct processes. One potential indicator of this distinction is the possibility that texture statistics play a different role in the figure vs. in the ground. To determine whether this is the case, we probed figure-ground processing with a family of local image statistics that specified textures that varied in the strength and spatial scale of structure, and the extent to which features are oriented. For image statistics that generated approximately isotropic textures, the threshold for identification of figure-ground structure was determined by the difference in correlation strength in figure vs. ground, independent of whether the correlations were present in figure, ground, or both. However, for image statistics with strong orientation content, thresholds were up to two times higher for correlations in the ground, vs. the figure. This held equally for texture-defined objects with convex or concave boundaries, indicating that these threshold differences are driven by border ownership, not boundary shape. Similar threshold differences were found for presentation times ranging from 125 to 500 ms. These findings identify a qualitative difference in how texture is used for figure-ground analysis, vs. texture discrimination. Additionally, it reveals a functional recursion: texture differences are needed to identify tentative boundaries and consequent scene organization into figure and ground, but then scene organization modifies sensitivity to texture differences according to the figure-ground assignment.

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Similarity judgments are commonly used to study mental representations and their neural correlates. This approach has been used to characterize perceptual spaces in many domains: colors, objects, images, words, and sounds. Ideally, one might want to compare estimates of perceived similarity between all pairs of stimuli, but this is often impractical. For example, if one asks a subject to compare the similarity of two items with the similarity of two other items, the number of comparisons grows with the fourth power of the stimulus set size. An alternative strategy is to ask a subject to rate similarities of isolated pairs, e.g., on a Likert scale. This is much more efficient (the number of ratings grows quadratically with set size rather than quartically), but these ratings tend to be unstable and have limited resolution, and the approach also assumes that there are no context effects. Here, a novel ranking paradigm for efficient collection of similarity judgments is presented, along with an analysis pipeline (software provided) that tests whether Euclidean distance models account for the data. Typical trials consist of eight stimuli around a central reference stimulus: the subject ranks stimuli in order of their similarity to the reference. By judicious selection of combinations of stimuli used in each trial, the approach has internal controls for consistency and context effects. The approach was validated for stimuli drawn from Euclidean spaces of up to five dimensions. The approach is illustrated with an experiment measuring similarities among 37 words. Each trial yields the results of 28 pairwise comparisons of the form, "Was A more similar to the reference than B was to the reference?" While directly comparing all pairs of pairs of stimuli would have required 221445 trials, this design enables reconstruction of the perceptual space from 5994 such comparisons obtained from 222 trials.

Learning complex texture discrimination

Higher-order spatial correlations contribute strongly to visual structure and salience, and are common in the natural environment. One method for studying this structure has been through the use of highly controlled texture patterns whose obvious structure is defined entirely by third- and higher-order correlations. Here we examine the effects that longer-term training has on discrimination of 17 such texture types. Training took place in 14 sessions over 42 days. Discrimination performance increased at different rates for different textures. The time required to complete a visit reduced by 25.4% (p=0.0004). Factor analysis was applied to data from the learning and experienced phases of the experiment. This indicated that the gain in speed was accompanied by an increase in the number of mechanisms contributing to discrimination. Learning was not affected by sleep quality but was affected by extreme tiredness (p<0.01). The improved discrimination and speed were retained for 2.5 months. Overall, the effects were consistent with perceptual learning. The observed learning is likely related to the adaptation of innate mechanisms that underlie our ability to identify nonredundant, visually salient structure in natural images. It may involve cortical V2 and appears to involve increased strength, speed, and breadth of connections within our internal representation of this complex perceptual space.

Efficient coding of natural scene statistics predicts discrimination thresholds for grayscale textures

Previously, in Hermundstad et al., 2014, we showed that when sampling is limiting, the efficient coding principle leads to a 'variance is salience' hypothesis, and that this hypothesis accounts for visual sensitivity to binary image statistics. Here, using extensive new psychophysical data and image analysis, we show that this hypothesis accounts for visual sensitivity to a large set of grayscale image statistics at a striking level of detail, and also identify the limits of the prediction. We define a 66-dimensional space of local grayscale light-intensity correlations, and measure the relevance of each direction to natural scenes. The 'variance is salience' hypothesis predicts that two-point correlations are most salient, and predicts their relative salience. We tested these predictions in a texture-segregation task using un-natural, synthetic textures. As predicted, correlations beyond second order are not salient, and predicted thresholds for over 300 second-order correlations match psychophysical thresholds closely (median fractional error <0.13).

The multidimensional brain

Brain activity takes place in three spatial-plus time dimensions. This rather obvious claim has been recently questioned by papers that, taking into account the big data outburst and novel available computational tools, are starting to unveil a more intricate state of affairs. Indeed, various brain activities and their correlated mental functions can be assessed in terms of trajectories embedded in phase spaces of dimensions higher than the canonical ones. In this review, I show how further dimensions may not just represent a convenient methodological tool that allows a better mathematical treatment of otherwise elusive cortical activities, but may also reflect genuine functional or anatomical relationships among real nervous functions. I then describe how to extract hidden multidimensional information from real or artificial neurodata series, and make clear how our mind dilutes, rather than concentrates as currently believed, inputs coming from the environment. Finally, I argue that the principle "the higher the dimension, the greater the information" may explain the occurrence of mental activities and elucidate the mechanisms of human diseases associated with dimensionality reduction.

Image segmentation driven by elements of form

While luminance, contrast, orientation, and terminators are well-established features that are extracted in early visual processing and support the parsing of an image into its component regions, the role of more complex features, such as closure and convexity, is less clear. A main barrier in understanding the roles of such features is that manipulating their occurrence typically entails changes in the occurrence of more elementary features as well. To address this problem, we developed a set of synthetic visual textures, constructed by replacing the binary coloring of standard maximum-entropy textures with tokens (tiles) containing curved or angled elements. The tokens were designed so that there were no discontinuities at their edges, and so that changing the correlation structure of the underlying binary texture changed the shapes that were produced. The resulting textures were then used in psychophysical studies, demonstrating that the resulting feature differences sufficed to drive segmentation. However, in contrast to previous findings for lower-level features, sensitivities to increases and decreases of feature occurrence were unequal. Moreover, the texture-segregation response depended on the kind of token (curved vs. angular, filled-in vs. outlined), and not just on the correlation structure. Analysis of this dependence indicated that simple closed contours and convex elements suffice to drive image segmentation, in the absence of changes in lower-level cues.

Textures as Probes of Visual Processing

Visual textures are a class of stimuli with properties that make them well suited for addressing general questions about visual function at the levels of behavior and neural mechanism. They have structure across multiple spatial scales, they put the focus on the inferential nature of visual processing, and they help bridge the gap between stimuli that are analytically convenient and the complex, naturalistic stimuli that have the greatest biological relevance. Key questions that are well suited for analysis via visual textures include the nature and structure of perceptual spaces, modulation of early visual processing by task, and the transformation of sensory stimuli into patterns of population activity that are relevant to perception.