Estimating multiple temporal mechanisms in human vision

Visual adaptation alters the apparent speed of real-world actions

The apparent physical speed of an object in the field of view remains constant despite variations in retinal velocity due to viewing conditions (velocity constancy). For example, people and cars appear to move across the field of view at the same objective speed regardless of distance. In this study a series of experiments investigated the visual processes underpinning judgements of objective speed using an adaptation paradigm and video recordings of natural human locomotion. Viewing a video played in slow-motion for 30 seconds caused participants to perceive subsequently viewed clips played at standard speed as too fast, so playback had to be slowed down in order for it to appear natural; conversely after viewing fast-forward videos for 30 seconds, playback had to be speeded up in order to appear natural. The perceived speed of locomotion shifted towards the speed depicted in the adapting video (‘re-normalisation’). Results were qualitatively different from those obtained in previously reported studies of retinal velocity adaptation. Adapting videos that were scrambled to remove recognizable human figures or coherent motion caused significant, though smaller shifts in apparent locomotion speed, indicating that both low-level and high-level visual properties of the adapting stimulus contributed to the changes in apparent speed.

Velocity perception in a moving observer

Previous research has shown that when a moving stimulus is presented to a moving observer, the perceived speed of the stimulus is affected by vestibular self-motion signals (Hogendoorn, Verstraten, MacDougall, & Alais, 2017. Vision Research 130, 22-30.). This interaction was interpreted as a weighted sum of visual and vestibular motion signals. This interpretation also predicts effects of vestibular self-motion signals on perceived speed. Here, we test this prediction in two experiments. In Experiment 1, moving observers carried out a visual speed discrimination task in order to establish points of subjective equality (PSE) between stimuli presented in the same or opposite direction of self-motion. We observed robust effects of self-motion on perceived speed, with self-motion in the same direction as visual motion resulting in increases in perceived speed and vice versa. These effects were well- described by a limited-width integration window. In Experiment 2, the same observers carried out another speed discrimination task in order to establish discrimination thresholds. According to the Weber-Fechner law, these thresholds are expected to increase or decrease along with perceived speed. However, no effect of self-motion on discrimination thresholds was observed. This pattern of results suggests a limit on speed discrimination performance early in the visual system, with visuo-vestibular integration in later downstream areas. These results are consistent with previous work on heading perception.

The Effect of Spatial-Frequency Filtering on the Visual Processing of Global Structure

In three experiments we measured reaction times (RTs) and error rates in identifying the global structure of spatially filtered stimuli whose spatial-frequency content was selected by means of three types of 2-D isotropic filters (Butterworth of order 2, Butterworth of order 10, and a filters with total or partial Gaussian spectral profile). In each experiment, low-pass (LP), band-pass (BP), and high-pass (HP) filtered stimuli, with nine centre or cut-off spatial frequencies, were used. Irrespective of the type of filter, the experimental results showed that: (a) RTs to stimuli with low spatial frequencies were shorter than those to stimuli with medium or high spatial frequencies, (b) RTs to LP filtered stimuli were nearly constant, but they increased in a non-monotonic way with the filter centre spatial frequency in BP filtered stimuli and with the filter cut-off frequency in HP filtered stimuli, and (c) the identification of the global pattern occurred with all visible stimuli used, including BP and HP images without low spatial frequencies. To remove the possible influence of the energy, a fourth experiment was conducted with Gaussian filtered stimuli of equal contrast power ( crms = 0.065). Similar results to those described above were found for stimuli with spatial-frequency content higher than 2 cycles deg−1. A model of isotropic first-order visual channels collecting the stimulus spectral energy in all orientations explains the RT data. A subsequent second-order nonlinear amplitude demodulation process, applied to the output of the most energetic first-order channel, could explain the perception of global structure of each spatially filtered stimulus, including images lacking low spatial frequencies.

Temporal frequency of flickering-distortion optimized video halftoning for electronic paper

Video halftoning is a key technology for use in electronic paper (e-paper) or smart paper, which is an emerging display device that has received considerable attention recently. In this paper, a temporal frequency of flickering-distortion optimized video halftoning method is proposed. We first uncover three visual defects that conventional neighboring frame referencing-based video halftoning methods, due to their sequential changes of reference frames, will encounter. To deal with the problem, we then propose a reference frame update per GOP-based error diffusion video halftoning method based on a flickering sensitivity-based human visual model. To efficiently compromise between average temporal frequency of flickering (ATFoF) and visual quality, temporal frequency of flickering-distortion (TFoFD) is presented as a metric for video halftoning performance evaluation. Based on the proposed probability model of video halftoning, the TFoFD curve can be accurately estimated to optimize the tradeoff between quality and ATFoF before the video is halftoned. Our temporal frequency of flickering-distortion optimization strategy can also be applied to other video halftoning schemes for performance improvement. Experimental results and comparisons with known methods demonstrate the effectiveness of our video halftoning method.

Estimating the growth of internal evidence guiding perceptual decisions

Perceptual decision-making is thought to involve a gradual accrual of noisy evidence. Temporal integration of the evidence reduces the relative contribution of dynamic internal noise to the decision variable, thereby boosting its signal-to-noise ratio. We aimed to estimate the internal evidence guiding perceptual decisions over time, using a novel combination of external noise and the response signal methods. Observers performed orientation discrimination of patterns presented in external noise. We varied the contrast of the patterns and the delay at which observers were forced to signal their decision. Each test stimulus (patterns and noise sample) was presented twice. Across two experiments we varied the availability of the visual stimulus for processing. Observer model analyses of discrimination accuracy and response consistency to two passes of the same stimulus, suggested that there was very little growth in the internal evidence. The improvement in accuracy over time characterised by the speed-accuracy trade-off function predominantly reflected a decreasing proportion of non-visual decisions, or pure guesses. There was no advantage to having the visual patterns visible for longer than 80 ms, indicating that only the visual information in a short window after display onset was used to drive the decisions. The remarkable constancy of the internal evidence over time suggests that temporal integration of the sensory information was very limited. Alternatively, more extended integration of the evidence from memory could have taken place, provided that the dominant source of internal noise limiting performance occurs between-trials, which cannot be reduced by prolonged evidence integration.

The horizontal effect in suppression: Anisotropic overlay and surround suppression at high and low speeds

When a pattern of broad spatial content is viewed by an observer, the multiple spatial components in the pattern stimulate detecting-mechanisms that suppress each other. This suppression is anisotropic, being relatively greater at horizontal, and least at obliques (the "horizontal effect"). Here, suppression of a grating by a naturalistic (1/f) broadband mask is shown to be larger when the broadband masks are temporally similar to the target's temporal properties, and generally anisotropic, with the anisotropy present across all spatio-temporal parings tested. We also show that both suppression from within the region of the test pattern (overlay suppression) and from outside of this region (surround suppression) show the horizontal-effect anisotropy. We conclude that these suppression effects stem from locally-tuned and anisotropically-weighted gain-control pools.

Distinct perceptual grouping pathways revealed by temporal carriers and envelopes

S. E. Guttman, L. A. Gilroy, and R. Blake (2005) investigated whether observers could perform temporal grouping in multi-element displays where each local element was stochastically modulated over time along one of several potential dimensions--or "messenger types"--such as contrast, position, orientation, or spatial scale. Guttman et al.'s data revealed that grouping discards messenger type and therefore support a single-pathway model that groups elements with similar temporal waveforms. In the current study, we carried out three experiments in which temporal-grouping information resided either in the carrier, the envelope, or the combined carrier and envelope of each messenger's timecourse. Results revealed that grouping is highly specific for messenger type if carrier envelopes lack grouping information but largely messenger nonspecific if carrier envelopes contain grouping information. These imply that temporal grouping is mediated by several messenger-specific carrier pathways as well as by a messenger-nonspecific envelope pathways. Findings also challenge simple temporal-filtering accounts of perceptual grouping (E. H. Adelson & H. Farid, 1999).

A static geometrical illusion contributes largely to the footsteps illusion

When a black and a white rectangle drifts across a stationary striped background with constant velocity, the rectangles appear to alternately speed up and slow down. Anstis (2001, Perception 30 785-794; 2004, Vision Research 44 2171-2178) suggested that this 'footsteps' illusion is due to confusion between contrast and velocity signaling in the motion detectors of the human visual system. To test this explanation, three experiments were carried out. In experiment 1, the magnitudes of the footsteps illusion in dynamic and static conditions was compared. If motion detectors play an important role in causing the illusion, it should be reduced in the static condition. Remarkably, however, we found that the illusory misalignment between the black and the white rectangle was even more prominent in the static condition than in the dynamic condition. In experiment 2, we measured the temporal-frequency properties of the footsteps illusion. The results showed that the footsteps illusion was tuned to low temporal frequencies. This suggests that the static illusory misalignment can contribute sufficiently to the dynamic illusory misalignment. In experiment 3, the magnitude of the illusion was measured with the rectangles drifting on a temporally modulated background instead of a spatially modulated background. If contrast affects the apparent velocity of the rectangles, temporal modulation of a uniform background should also cause the footsteps illusion. However, the results showed that the magnitude of the illusion was much reduced in this condition. Taken together, the results indicate that the footsteps illusion can be regarded as a static geometrical illusion induced by the striped background and that motion detectors play a minor role at best.

Spatial versus temporal grouping in a modified Ternus display

The Ternus display can induce a percept of 'element motion' or 'group motion'. Conventionally, this has been attributed to two different motion processes, with different spatial and temporal ranges. In contrast, recent studies have emphasised spatial and temporal grouping principles as underlying the apparent motion percepts in the Ternus display. The present study explored effects of spatial and temporal grouping on the apparent motion percept in a novel Ternus display of oriented Gabor elements with no inter-frame interval. Each frame of this stimulus could be further divided into 'sub-frames', and the orientation of the carriers was changed across these sub-frames. In four experiments transitions were found between the motion percepts with changes in orientation across time (Experiment 1) and space (Experiment 2), and with a temporal offset in the orientation change of the outer element (Experiment 3) to the extent that group motion was not perceived even with large orientation changes over time that previously led to group motion (Experiment 4). Collectively, these results indicate that while spatial properties have an influence in determining the percept of the Ternus display, temporal properties also have a strong influence, and can override spatial grouping. However, these temporal effects cannot be attributed to spatio-temporal limits of low-level motion processes. Some aspects of the observed spatial grouping effects can be accounted for in terms of a modified association field, which may occur through connectivity of orientation selective units in V1. The temporal effects observed are considered in terms of temporal integration, the transitional value at a temporal offset of 40ms being remarkably similar to psychophysical and neurophysiological estimates of the peak temporal impulse response. These temporal responses could be detected at a higher level in the system, providing a basis for apparent motion perception.

Video halftoning

This paper studies video halftoning that renders a digital video sequence onto display devices, which have limited intensity resolutions and color palettes, by trading the spatiotemporal resolution for enhanced intensity/color resolution. This trade is needed when a continuous tone video is not necessary or not practical for video display, transmission, and storage. In particular, the quantization error of a pixel is diffused to its spatiotemporal neighbors by separable one-dimensional temporal and two-dimensional spatial error diffusions. Motion-adaptive gain control is employed to enhance the temporal consistency of the visual patterns by minimizing the flickering artifacts. Experimental results of halftone and colortone videos are demonstrated and evaluated with various halftoning techniques.

The temporal impulse response underlying saccadic decisions

Models of perceptual decision making often assume that sensory evidence is accumulated over time in favor of the various possible decisions, until the evidence in favor of one of them outweighs the evidence for the others. Saccadic eye movements are among the most frequent perceptual decisions that the human brain performs. We used stochastic visual stimuli to identify the temporal impulse response underlying saccadic eye movement decisions. Observers performed a contrast search task, with temporal variability in the visual signals. In experiment 1, we derived the temporal filter observers used to integrate the visual information. The integration window was restricted to the first approximately 100 ms after display onset. In experiment 2, we showed that observers cannot perform the task if there is no useful information to distinguish the target from the distractor within this time epoch. We conclude that (1) observers did not integrate sensory evidence up to a criterion level, (2) observers did not integrate visual information up to the start of the saccadic dead time, and (3) variability in saccade latency does not correspond to variability in the visual integration period. Instead, our results support a temporal filter model of saccadic decision making. The temporal impulse response identified by our methods corresponds well with estimates of integration times of V1 output neurons.

HVS-based medical image compression

INTRODUCTION

With the promotion and application of digital imaging technology in the medical domain, the amount of medical images has grown rapidly. However, the commonly used compression methods cannot acquire satisfying results.

METHODS

In this paper, according to the existed and stated experiments and conclusions, the lifting step approach is used for wavelet decomposition. The physical and anatomic structure of human vision is combined and the contrast sensitivity function (CSF) is introduced as the main research issue in human vision system (HVS), and then the main designing points of HVS model are presented. On the basis of multi-resolution analyses of wavelet transform, the paper applies HVS including the CSF characteristics to the inner correlation-removed transform and quantization in image and proposes a new HVS-based medical image compression model.

RESULTS

The experiments are done on the medical images including computed tomography (CT) and magnetic resonance imaging (MRI). At the same bit rate, the performance of SPIHT, with respect to the PSNR metric, is significantly higher than that of our algorithm. But the visual quality of the SPIHT-compressed image is roughly the same as that of the image compressed with our approach. Our algorithm obtains the same visual quality at lower bit rates and the coding/decoding time is less than that of SPIHT.

CONCLUSIONS

The results show that under common objective conditions, our compression algorithm can achieve better subjective visual quality, and performs better than that of SPIHT in the aspects of compression ratios and coding/decoding time.

Visual adjustments to temporal blur

After observers have adapted to an edge that is spatially blurred or sharpened, a focused edge appears too sharp or blurred, respectively. These adjustments to blur may play an important role in calibrating spatial sensitivity. We examined whether similar adjustments influence the perception of temporal edges, by measuring the appearance of a step change in the luminance of a uniform field after adapting to blurred or sharpened transitions. Stimuli were square-wave alternations (at 1 to 8 Hz) filtered by changing the slope of the amplitude spectrum. A two-alternative-forced-choice task was used to adjust the slope until it appeared as a step change, or until it matched the perceived transitions in a reference stimulus. Observers could accurately set the waveform to a square wave, but only at the slower alternation rates. However, these settings were strongly biased by prior adaptation to filtered stimuli, or when the stimuli were viewed within temporally filtered surrounds. Control experiments suggest that the latter induction effects result directly from the temporal blur and are not simply a consequence of brightness induction in the fields. These results suggest that adaptation and induction adjust visual coding so that images are focused not only in space but also in time.

The motion aftereffect of transparent motion: Two temporal channels account for perceived direction

Adaptation to orthogonal transparent patterns drifting at the same speed produces a unidirectional motion aftereffect (MAE) whose direction is opposite the average adaptation direction. If the patterns move at different speeds, MAE direction can be predicted by an inverse vector average, using the observer's motion sensitivity to each individual pattern as vector magnitudes. These weights are well approximated by the duration of each pattern's MAE, as measured with static test patterns. However, previous efforts to use the inverse-vector-average rule with dynamic test patterns have failed. Generally, these studies have used spatially and temporally broadband test stimuli. Here, in order to gain insight into the possible contribution of temporal channels, we filtered our test pattern in the temporal domain to produce five ideal, octave-width pass-bands. MAE durations were measured for single-component stimuli drifting at various adaptation speeds and tested at a range of temporal frequencies. Then, two components with orthogonal directions and different speeds were combined and the direction of the resulting MAE was measured. The key findings are that: (i) for a given adaptation speed, the duration of a single component's MAE is dependent on test temporal frequency; (ii) the direction of MAEs produced by transparent motion (i.e., bivectorial adaptation) also varies strongly as a function test temporal frequency (by up to 90 degrees for some speed pairings); and (iii) the inverse-vector-average rule predicts the direction of the transparent MAE provided the MAE durations used to weight the vector combination were obtained from stimuli matched in adaptation speed and test temporal frequency. These results are discussed in terms of the number and shape of temporal channels in our visual system.

Temporal aspects of spatial interactions affecting stereo-matching solutions

Stereo-matching solutions minimize disparity relative to the horopter (minimum-absolute-disparity or MAD), and differences in disparity between adjacent features (minimum-relative-disparity or MRD). When placed in conflict, spatial proximity promotes MRD over MAD solutions. How does temporal proximity of neighboring features affect strength of these spatial interactions? We quantified the inter-stimulus interval (ISI) over which an unambiguous disparity pattern influenced stereo-matches for patterns with several possible solutions. Likelihood of MRD decreased as ISI increased (48.9 ms time constant) and increased as contrast was reduced for short ISIs, suggesting that monocular persistence (temporal impulse response) underlies the temporal interaction.

Temporal detection in human vision: dependence on eccentricity

Studies of human perception of time-varying luminance often aim to estimate either temporal impulse response shapes or temporal modulation transfer functions (MTFs) of putative temporal processing mechanisms. Previously, temporal masking data have been used to estimate the properties and numbers of these temporal mechanisms in central vision for 1 cycle per degree (cpd) targets [Fredericksen and Hess (1998)]. The same methods have been used to explore how these properties change with stimulus energy [Fredericksen and Hess (1997)] and spatial frequency [Fredericksen and Hess (1999)]. We present here analyses of the properties of temporal mechanisms that detect temporal variations of luminance in peripheral vision. The results indicate that a two-filter model provides the best model for our masking data, but that no multiple filter model provides an acceptable fit across the range of parameters varied in this study. Single-filter modelling shows differences between processing mechanisms at 1 cpd in central vision and those that operate eccentrically. We find evidence that this change is because of differences in relative sensitivities of the mechanisms, and to differences in fundamental mechanism impulse responses.

Techniques for fast stereoscopic MRI

Stereoscopic MRI can impart 3D perception with only two image acquisitions. This economy over standard multiplanar 3D volume renderings allows faster frame rates, which are needed for real-time imaging applications. Real-time 3D perception may enhance the appreciation of complex anatomical structures, and may improve hand-eye coordination while manipulating a medical device during an image-guided interventional procedure. To this goal, a system is being developed to acquire and display stereoscopic MR images in real-time. A clinically used, fast gradient-recalled echo-train sequence has been modified to produce stereo image pairs. Features have been added for depth cueing, view sharing, and bulk signal suppression. A workstation was attached to a clinical MR scanner for fast data extraction, image reconstruction and stereoscopic image display.

A bayesian model for the measurement of visual velocity

Several models have been proposed for how the brain measures velocity from the output of motion-energy units. These models make some unrealistic assumptions such as the use of Gabor-shaped temporal filters, which are non causal, or flat spatial spectra, which are invalidated by existing data. We present a Bayesian model of velocity perception, which makes more realistic assumptions and allows the estimation of local retinal velocity regardless of the specific mathematical form of the spatial and temporal filters used. The model is consistent with several aspects of speed perception, such as the dependence of perceived speed on contrast.

Flicker sensitivity as a function of target area with and without temporal noise

Flicker sensitivities (1-30 Hz) in foveal, photopic vision were measured as functions of stimulus area with and without strong external white temporal noise. Stimuli were circular, sinusoidally flickering sharp-edged spots of variable diameters (0.25-4 degrees ) but constant duration (2 s), surrounded by a uniform equiluminant field. The data was described with a model comprising (i) low-pass filtering in the retina (R), with a modulation transfer function (MTF) of a form derived from responses of cones; (ii) normalisation of the temporal luminance distribution by the average luminance; (iii) high-pass filtering by postreceptoral neural pathways (P), with an MTF proportional to temporal frequency; (iv) addition of internal white neural noise (N(i)); (v) integration over a spatial window; and (vi) detection by a suboptimal temporal matched filter of efficiency eta. In strong external noise, flicker sensitivity was independent of spot area. Without external noise, sensitivity increased with the square root of stimulus area (Piper's law) up to a critical area (A(c)), where it reaches a maximum level (S(max)). Both A(c) and eta were monotonic functions of temporal frequency (f), such that log A(c) increased and log eta decreased linearly with log f. Remarkably, the increase in spatial integration area and the decrease in efficiency were just balanced, so A(c)(f)eta(f) was invariant against f. Thus the bandpass characteristics of S(max)(f) directly reflected the composite effect of the distal filters R(f) and P(f). The temporal equivalent (N(it)) of internal neural noise (N(i)) decreased in inverse proportion to spot area up to A(c) and then stayed constant indicating that spatially homogeneous signals and noise are integrated over the same area.

The temporal properties of first- and second-order vision

Vision is sensitive to first-order modulations of luminance and second-order modulations of image contrast. There is now a body of evidence that the two types of modulation are detected by separate mechanisms. Some previous experiments on motion detection have suggested that the second-order system is quite sluggish compared to the first-order system. Here we derive temporal properties of first- and second-order vision at threshold from studies of temporal integration and two-pulse summation. Three types of modulation were tested: luminance gratings alone, luminance modulations added to dynamic visual noise, and contrast modulations of dynamic noise. Data from the two-pulse summation experiment were used to derive impulse response functions for the three types of stimulus. These were then used to predict performance in the temporal integration experiment. Temporal frequency response functions were obtained as the Fourier transform of impulse responses derived from data averaged across two observers. The response to noise-free luminance gratings of 2 c/deg was bi-phasic and transient in the time domain, and bandpass in the frequency domain. The addition of dynamic noise caused the response to become mono-phasic, sustained and low-pass. The response to contrast modulated noise (second-order) was also mono-phasic, sustained and low-pass, with only a slightly longer integration time than in the first-order case. The ultimate roll-off at high frequencies was about the same as for the first-order case. We conclude that second-order vision may not be as sluggish as previously thought.

A Bayesian model of temporal frequency masking

The data of Anderson and Burr [1985. Vision Research, 25, 1147-1154] on the temporal-frequency (TF) specificity of noise maskers indicate that the effect of TF masking is broad and varies across spatial frequency (SF) channels. One subtle but significant feature of the data is that the TF at which the effect of masking is maximal falls continuously as the test TF falls. This continuous shift is hard to reconcile with models of detection in the literature that relate detection to the most sensitive filter, without resorting to a large number of temporal filters. We developed a new model, which relies on only three temporal filters and posits that detection is the result of a threshold decision based on the compound Bayesian probability of all filter responses, not just the most sensitive filter.

Temporal detection in human vision: dependence on spatial frequency

In the study of perception of temporal changes in luminance, it is customary to model perceptual performance as based on one or more linear filters. The task is then to estimate the temporal impulse responses or the representation of the impulse response in the frequency domain. Previously, temporal masking data have been used to estimate the properties and numbers of these temporal mechanisms (filters) in central vision for 1-cycle-per-degree (cpd) targets [Vision Res. 38, 1023 (1998)]. The same methods have been used to explore how properties of the estimated filters change with stimulus contrast energy [J. Opt. Soc. Am. A 14, 2557 (1997)]. We present estimated properties for temporal mechanisms that detect low spatial-frequency patterns. The results indicate that two filters provide the best model for performance when mask contrast is significant. There are also differences between properties for mechanisms that detect signal spatial frequencies of 1 cpd and 1/3 cpd. The sensitivity of the low-pass mechanism relative to the bandpass mechanism is reduced at 1/3 cpd, consistent with previous findings.

Characterizing human perceptual inefficiencies with equivalent internal noise

A widely used method for characterizing and comparing inefficiencies in perceptual processes is the method of equivalent internal noise--the amount of random internal noise necessary to produce the degree of inefficiency exhibited by the perceptual system in processing [J. Opt. Soc. Am. 46, 634 (1956)]. One normally estimates the amount of equivalent internal noise by systematically increasing the amount of external noise added to the signal stimulus and observing how threshold--signal stimulus energy required for an observer to maintain a given performance level--depends on the amount of external noise. In a variety of perceptual tasks, a simple noisy linear amplifier model [D. Pelli, Ph.D. dissertation (University of Cambridge, Cambridge, UK 1981)] has been utilized to estimate the equivalent internal noise Ninternal by fitting of the relation between threshold contrast c tau and external noise N(ext) at a single (d') performance level: c tau 2 =(d'/beta)2(N(ext)2 + Ninternal2). This model makes a strong prediction: Independent of observer and external noise contrast, the ratio between two thresholds at each external noise level is equal to the ratio of the two corresponding d' values. To our knowledge, this potential test for the internal consistency of the model had never been examined previously. In this study we estimated threshold ratios between multiple performance levels at various external noise contrasts in two different experiments: Gabor orientation identification, and Gabor detection. We found that, in both identification and detection, the observed threshold ratios between different performance levels departed substantially from the d' ratio predicted by the simple noisy linear amplifier model. An elaborated perceptual template model [Vision Res. 38, 1183 (1998)] with nonlinear transducer functions and multiplicative noise in addition to the additive noise in the simple linear amplifier model leads to a substantially better description of the data and suggests a reinterpretation of earlier results that relied on the simple noisy linear amplifier model. The relationship of our model and method to other recent parallel and independent developments [J. Opt. Soc. Am. A 14, 2406 (1997)] is discussed.