Local luminance and contrast in natural images

Higher-contrast images are better remembered during naturalistic encoding

It is unclear whether memory for images of poorer visibility (as low contrast or small size) will be lower due to weak signals elicited in early visual processing stages, or perhaps better since their processing may entail top-down processes (as effort and attention) associated with deeper encoding. We have recently shown that during naturalistic encoding (free viewing without task-related modulations), for image sizes between 3°-24°, bigger images stimulating more visual system processing resources at early processing stages are better remembered. Similar to size, higher contrast leads to higher activity in early visual processing. Therefore, here we hypothesized that during naturalistic encoding, at critical visibility ranges, higher contrast images will lead to higher signal-to-noise ratio and better signal quality flowing downstream and will thus be better remembered. Indeed, we found that during naturalistic encoding higher contrast images were remembered better than lower contrast ones (~ 15% higher accuracy, ~ 1.58 times better) for images at 7.5-60 RMS contrast range. Although image contrast and size modulate early visual processing very differently, our results further substantiate that at poor visibility ranges, during naturalistic non-instructed visual behavior, physical image dimensions (contributing to image visibility) impact image memory.

Neural correlates of perceptual similarity masking in primate V1

Visual detection is a fundamental natural task. Detection becomes more challenging as the similarity between the target and the background in which it is embedded increases, a phenomenon termed 'similarity masking'. To test the hypothesis that V1 contributes to similarity masking, we used voltage sensitive dye imaging (VSDI) to measure V1 population responses while macaque monkeys performed a detection task under varying levels of target-background similarity. Paradoxically, we find that during an initial transient phase, V1 responses to the target are enhanced, rather than suppressed, by target-background similarity. This effect reverses in the second phase of the response, so that in this phase V1 signals are positively correlated with the behavioral effect of similarity. Finally, we show that a simple model with delayed divisive normalization can qualitatively account for our findings. Overall, our results support the hypothesis that a nonlinear gain control mechanism in V1 contributes to perceptual similarity masking.

Predictably manipulating photoreceptor light responses to reveal their role in downstream visual responses

Computation in neural circuits relies on judicious use of nonlinear circuit components. In many cases, multiple nonlinear components work collectively to control circuit outputs. Separating the contributions of these different components is difficult, and this hampers our understanding of the mechanistic basis of many important computations. Here, we introduce a tool that permits the design of light stimuli that predictably alter rod and cone phototransduction currents - including stimuli that compensate for nonlinear properties such as light adaptation. This tool, based on well-established models for the rod and cone phototransduction cascade, permits the separation of nonlinearities in phototransduction from those in downstream circuits. This will allow, for example, direct tests of how adaptation in rod and cone phototransduction affects downstream visual signals and perception.

The influence of natural image statistics on upright orientation judgements

Humans have well-documented priors for many features present in nature that guide visual perception. Despite being putatively grounded in the statistical regularities of the environment, scene priors are frequently violated due to the inherent variability of visual features from one scene to the next. However, these repeated violations do not appreciably challenge visuo-cognitive function, necessitating the broad use of priors in conjunction with context-specific information. We investigated the trade-off between participants' internal expectations formed from both longer-term priors and those formed from immediate contextual information using a perceptual inference task and naturalistic stimuli. Notably, our task required participants to make perceptual inferences about naturalistic images using their own internal criteria, rather than making comparative judgements. Nonetheless, we show that observers' performance is well approximated by a model that makes inferences using a prior for low-level image statistics, aggregated over many images. We further show that the dependence on this prior is rapidly re-weighted against contextual information, even when misleading. Our results therefore provide insight into how apparent high-level interpretations of scene appearances follow from the most basic of perceptual processes, which are grounded in the statistics of natural images.

NEURAL CORRELATES OF PERCEPTUAL SIMILARITY MASKING IN PRIMATE V1

Visual detection is a fundamental natural task. Detection becomes more challenging as the similarity between the target and the background in which it is embedded increases, a phenomenon termed "similarity masking". To test the hypothesis that V1 contributes to similarity masking, we used voltage sensitive dye imaging (VSDI) to measure V1 population responses while macaque monkeys performed a detection task under varying levels of target-background similarity. Paradoxically, we find that during an initial transient phase, V1 responses to the target are enhanced, rather than suppressed, by target-background similarity. This effect reverses in the second phase of the response, so that in this phase V1 signals are positively correlated with the behavioral effect of similarity. Finally, we show that a simple model with delayed divisive normalization can qualitatively account for our findings. Overall, our results support the hypothesis that a nonlinear gain control mechanism in V1 contributes to perceptual similarity masking.

Effects of target-amplitude and background-contrast uncertainty predicted by a normalized template-matching observer

When detecting targets under natural conditions, the visual system almost always faces multiple, simultaneous, dimensions of extrinsic uncertainty. This study focused on the simultaneous uncertainty about target amplitude and background contrast. These dimensions have a large effect on detection and vary greatly in natural scenes. We measured the human performance for detecting a sine-wave target in white noise and natural-scene backgrounds for two levels of prior probability of the target being present. We derived and tested the ideal observer for white-noise backgrounds, a special case of a template-matching observer that dynamically moves its criterion with the background contrast (the DTM observer) and two simpler models with a fixed criterion: the template-matching (TM) observer and the normalized template-matching (NTM) observer that normalizes template response by background contrast. Simulations show that, when the target prior is low, the performance of the NTM observer is near optimal and the TM observer is near chance, suggesting that manipulating the target prior is valuable for distinguishing among models. Surprisingly, we found that the NTM and DTM observers better explain human performance than the TM observer for both target priors in both background types. We argue that the visual system most likely exploits contrast normalization, rather than dynamic criterion adjustment, to deal with simultaneous background contrast and target amplitude uncertainty. Finally, our findings show that the data collected under high levels of uncertainty have a rich structure capable of discriminating between models, providing an alternative approach for studying high dimensions of uncertainty.

Long-timescale anti-directional rotation in Drosophila optomotor behavior

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied Drosophila melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such 'anti-directional turning' is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

IRIS: Integrated Retinal Functionality in Image Sensors

Neuromorphic image sensors draw inspiration from the biological retina to implement visual computations in electronic hardware. Gain control in phototransduction and temporal differentiation at the first retinal synapse inspired the first generation of neuromorphic sensors, but processing in downstream retinal circuits, much of which has been discovered in the past decade, has not been implemented in image sensor technology. We present a technology-circuit co-design solution that implements two motion computations-object motion sensitivity and looming detection-at the retina's output that could have wide applications for vision-based decision-making in dynamic environments. Our simulations on Globalfoundries 22 nm technology node show that the proposed retina-inspired circuits can be fabricated on image sensing platforms in existing semiconductor foundries by taking advantage of the recent advances in semiconductor chip stacking technology. Integrated Retinal Functionality in Image Sensors (IRIS) technology could drive advances in machine vision applications that demand energy-efficient and low-bandwidth real-time decision-making.

Spatiotemporal visual statistics of aquatic environments in the natural habitats of zebrafish

Animal sensory systems are tightly adapted to the demands of their environment. In the visual domain, research has shown that many species have circuits and systems that exploit statistical regularities in natural visual signals. The zebrafish is a popular model animal in visual neuroscience, but relatively little quantitative data is available about the visual properties of the aquatic habitats where zebrafish reside, as compared to terrestrial environments. Improving our understanding of the visual demands of the aquatic habitats of zebrafish can enhance the insights about sensory neuroscience yielded by this model system. We analyzed a video dataset of zebrafish habitats captured by a stationary camera and compared this dataset to videos of terrestrial scenes in the same geographic area. Our analysis of the spatiotemporal structure in these videos suggests that zebrafish habitats are characterized by low visual contrast and strong motion when compared to terrestrial environments. Similar to terrestrial environments, zebrafish habitats tended to be dominated by dark contrasts, particularly in the lower visual field. We discuss how these properties of the visual environment can inform the study of zebrafish visual behavior and neural processing and, by extension, can inform our understanding of the vertebrate brain.

Insect vision: Contrast perception under fluctuating light

Natural light levels vary tremendously, both over the day and from minute to minute, creating a formidable challenge for animals that rely on vision to survive. New work in fruit flies demonstrates the neural mechanisms that produce luminance-invariant perceptions of visual contrast.

Fast visual adaptation to dim light in a cavity-nesting bird

Many birds move fast into dark nest cavities forcing the visual system to adapt to low light intensities. Their visual system takes between 15 and 60 min for complete dark adaptation, but little is known about the visual performance of birds during the first seconds in low light intensities. In a forced two-choice behavioural experiment we studied how well budgerigars can discriminate stimuli of different luminance directly after entering a darker environment. The birds made their choices within about 1 s and did not wait to adapt their visual system to the low light intensities. When moving from a bright facility into an environment with 0.5 log unit lower illuminance, the budgerigars detected targets with a luminance of 0.825 cd m^-2 on a black background. When moving into an environment with 1.7 or 3.5 log units lower illuminance, they detected targets with luminances between 0.106 and 0.136 cd m^-2. In tests with two simultaneously displayed targets, the birds discriminated similar luminance differences between the targets (Weber fraction of 0.41-0.54) in all light levels. Our results support the notion that partial adaptation of bird eyes to the lower illumination occurring within 1 s allows them to safely detect and feed their chicks.

Long timescale anti-directional rotation in Drosophila optomotor behavior

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied D. melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such "anti-directional turning" is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

Image enhancement and field of view enlargement in digital lensless holographic microscopy by multi-shot imaging

A method to improve the quality of reconstructed images while the field of view (FOV) is enlarged in digital lensless holographic microscopy (DLHM) is presented. Multiple DLHM holograms are recorded while a still sample is located at different places of the plane containing it. The different locations of the sample must produce a set of DLHM holograms that share an overlapped area with a fixed DLHM hologram. The relative displacement among multiple DLHM holograms is computed by means of a normalized cross-correlation. The value of the computed displacement is utilized to produce a new DLHM hologram resulting from the coordinated addition of multi-shot DLHM holograms with the corresponding compensated displacement. The composed DLHM hologram carries enhanced information of the sample in a larger format, leading to a reconstructed image with improved quality and larger FOV. The feasibility of the method is illustrated and validated with results obtained from imaging a calibration test target and a biological specimen.

Contrast and Luminance Gain Control in the Macaque's Lateral Geniculate Nucleus

There is substantial variation in the mean and variance of light levels (luminance and contrast) in natural visual scenes. Retinal ganglion cells maintain their sensitivity despite this variation using two adaptive mechanisms, which control how responses depend on luminance and on contrast. However, the nature of each mechanism and their interactions downstream of the retina are unknown. We recorded neurons in the magnocellular and parvocellular layers of the lateral geniculate nucleus (LGN) in anesthetized adult male macaques and characterized how their responses adapt to changes in contrast and luminance. As contrast increases, neurons in the magnocellular layers maintain sensitivity to high temporal frequency stimuli but attenuate sensitivity to low-temporal frequency stimuli. Neurons in the parvocellular layers do not adapt to changes in contrast. As luminance increases, both magnocellular and parvocellular cells increase their sensitivity to high-temporal frequency stimuli. Adaptation to luminance is independent of adaptation to contrast, as previously reported for LGN neurons in the cat. Our results are similar to those previously reported for macaque retinal ganglion cells, suggesting that adaptation to luminance and contrast result from two independent mechanisms that are retinal in origin.

Nonequilibrium dynamics of adaptation in sensory systems

Adaptation is used by biological sensory systems to respond to a wide range of environmental signals, by adapting their response properties to the statistics of the stimulus in order to maximize information transmission. We derive rules of optimal adaptation to changes in the mean and variance of a continuous stimulus in terms of Bayesian filters and map them onto stochastic equations that couple the state of the environment to an internal variable controlling the response function. We calculate numerical and exact results for the speed and accuracy of adaptation and its impact on information transmission. We find that, in the regime of efficient adaptation, the speed of adaptation scales sublinearly with the rate of change of the environment. Finally, we exploit the mathematical equivalence between adaptation and stochastic thermodynamics to quantitatively relate adaptation to the irreversibility of the adaptation time course, defined by the rate of entropy production. Our results suggest a means to empirically quantify adaptation in a model-free and nonparametric way.

Homeostatic plasticity in the retina

Vision begins in the retina, whose intricate neural circuits extract salient features of the environment from the light entering our eyes. Neurodegenerative diseases of the retina (e.g., inherited retinal degenerations, age-related macular degeneration, and glaucoma) impair vision and cause blindness in a growing number of people worldwide. Increasing evidence indicates that homeostatic plasticity (i.e., the drive of a neural system to stabilize its function) can, in principle, preserve retinal function in the face of major perturbations, including neurodegeneration. Here, we review the circumstances and events that trigger homeostatic plasticity in the retina during development, sensory experience, and disease. We discuss the diverse mechanisms that cooperate to compensate and the set points and outcomes that homeostatic retinal plasticity stabilizes. Finally, we summarize the opportunities and challenges for unlocking the therapeutic potential of homeostatic plasticity. Homeostatic plasticity is fundamental to understanding retinal development and function and could be an important tool in the fight to preserve and restore vision.

Retinal Encoding of Natural Scenes

An ultimate goal in retina science is to understand how the neural circuit of the retina processes natural visual scenes. Yet most studies in laboratories have long been performed with simple, artificial visual stimuli such as full-field illumination, spots of light, or gratings. The underlying assumption is that the features of the retina thus identified carry over to the more complex scenario of natural scenes. As the application of corresponding natural settings is becoming more commonplace in experimental investigations, this assumption is being put to the test and opportunities arise to discover processing features that are triggered by specific aspects of natural scenes. Here, we review how natural stimuli have been used to probe, refine, and complement knowledge accumulated under simplified stimuli, and we discuss challenges and opportunities along the way toward a comprehensive understanding of the encoding of natural scenes. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Contrast discrimination in images of natural scenes

Contrast discrimination determines the threshold contrast required to distinguish between two suprathreshold visual stimuli. It is typically measured using sine-wave gratings. We first present a modification to Barten's semi-mechanistic contrast discrimination model to account for spatial frequency effects and demonstrate how the model can successfully predict visual thresholds obtained from published classical contrast discrimination studies. Contrast discrimination functions are then measured from images of natural scenes, using a psychophysical paradigm based on that employed in our previous study of contrast detection sensitivity. The proposed discrimination model modification is shown to successfully predict discrimination thresholds for structurally very different types of natural image stimuli. A comparison of results shows that, for normal contrast levels in natural scene viewing, contextual contrast detection and discrimination are approximately the same and almost independent of spatial frequency within the range of 1-20 c/deg. At higher frequencies, both sensitivities decrease in magnitude due to optical limitations of the eye. The results are discussed in relation to current image quality models.

Visual segmentation of complex naturalistic structures in an infant eye-tracking search task

An infant’s everyday visual environment is composed of a complex array of entities, some of which are well integrated into their surroundings. Although infants are already sensitive to some categories in their first year of life, it is not clear which visual information supports their detection of meaningful elements within naturalistic scenes. Here we investigated the impact of image characteristics on 8-month-olds’ search performance using a gaze contingent eye-tracking search task. Infants had to detect a target patch on a background image. The stimuli consisted of images taken from three categories: vegetation, non-living natural elements (e.g., stones), and manmade artifacts, for which we also assessed target background differences in lower- and higher-level visual properties. Our results showed that larger target-background differences in the statistical properties scaling invariance and entropy, and also stimulus backgrounds including low pictorial depth, predicted better detection performance. Furthermore, category membership only affected search performance if supported by luminance contrast. Data from an adult comparison group also indicated that infants’ search performance relied more on lower-order visual properties than adults. Taken together, these results suggest that infants use a combination of property- and category-related information to parse complex visual stimuli.

Luminance and Contrast of Images in the THINGS Database

The THINGS database is a freely available stimulus set that has the potential to facilitate the generation of theory that bridges multiple areas within cognitive neuroscience. The database consists of 26,107 high quality digital photos that are sorted into 1,854 concepts. While a valuable resource, relatively few technical details relevant to the design of studies in cognitive neuroscience have been described. We present an analysis of two key low-level properties of THINGS images, luminance and luminance contrast. These image statistics are known to influence common physiological and neural correlates of perceptual and cognitive processes. In general, we found that the distributions of luminance and contrast are in close agreement with the statistics of natural images reported previously. However, we found that image concepts are separable in their luminance and contrast: we show that luminance and contrast alone are sufficient to classify images into their concepts with above chance accuracy. We describe how these factors may confound studies using the THINGS images, and suggest simple controls that can be implemented a priori or post-hoc. We discuss the importance of using such natural images as stimuli in psychological research.

First-order visual interneurons distribute distinct contrast and luminance information across ON and OFF pathways to achieve stable behavior

The accurate processing of contrast is the basis for all visually guided behaviors. Visual scenes with rapidly changing illumination challenge contrast computation because photoreceptor adaptation is not fast enough to compensate for such changes. Yet, human perception of contrast is stable even when the visual environment is quickly changing, suggesting rapid post receptor luminance gain control. Similarly, in the fruit fly Drosophila, such gain control leads to luminance invariant behavior for moving OFF stimuli. Here, we show that behavioral responses to moving ON stimuli also utilize a luminance gain, and that ON-motion guided behavior depends on inputs from three first-order interneurons L1, L2, and L3. Each of these neurons encodes contrast and luminance differently and distributes information asymmetrically across both ON and OFF contrast-selective pathways. Behavioral responses to both ON and OFF stimuli rely on a luminance-based correction provided by L1 and L3, wherein L1 supports contrast computation linearly, and L3 non-linearly amplifies dim stimuli. Therefore, L1, L2, and L3 are not specific inputs to ON and OFF pathways but the lamina serves as a separate processing layer that distributes distinct luminance and contrast information across ON and OFF pathways to support behavior in varying conditions.

"Blue Sky Effect": Contextual Influences on Pupil Size During Naturalistic Visual Search

Pupil size is influenced by cognitive and non-cognitive factors. One of the strongest modulators of pupil size is scene luminance, which complicates studies of cognitive pupillometry in environments with complex patterns of visual stimulation. To help understand how dynamic visual scene statistics influence pupil size during an active visual search task in a visually rich 3D virtual environment (VE), we analyzed the correlation between pupil size and intensity changes of image pixels in the red, green, and blue (RGB) channels within a large window (~14 degrees) surrounding the gaze position over time. Overall, blue and green channels had a stronger influence on pupil size than the red channel. The correlation maps were not consistent with the hypothesis of a foveal bias for luminance, instead revealing a significant contextual effect, whereby pixels above the gaze point in the green/blue channels had a disproportionate impact on pupil size. We hypothesized this differential sensitivity of pupil responsiveness to blue light from above as a “blue sky effect,” and confirmed this finding with a follow-on experiment with a controlled laboratory task. Pupillary constrictions were significantly stronger when blue was presented above fixation (paired with luminance-matched gray on bottom) compared to below fixation. This effect was specific for the blue color channel and this stimulus orientation. These results highlight the differential sensitivity of pupillary responses to scene statistics in studies or applications that involve complex visual environments and suggest blue light as a predominant factor influencing pupil size.

Enhancing the dark side: asymmetric gain of cone photoreceptors underpins their discrimination of visual scenes based on skewness

Psychophysical data indicate that humans can discriminate visual scenes based on their skewness, i.e. the ratio of dark and bright patches within a visual scene. It has also been shown that at a phenomenological level this skew discrimination is described by the so-called blackshot mechanism, which accentuates strong negative contrasts within a scene. Here, we present a set of observations suggesting that the underlying computation might start as early as the cone phototransduction cascade, whose gain is higher for strong negative contrasts than for strong positive contrasts. We recorded from goldfish cone photoreceptors and found that the asymmetry in the phototransduction gain leads to responses with larger amplitudes when using negatively rather than positively skewed light stimuli. This asymmetry in amplitude was present in the cone photocurrent, voltage response and synaptic output. Given that the properties of the phototransduction cascade are universal across vertebrates, it is possible that the mechanism shown here gives rise to a general ability to discriminate between scenes based only on their skewness, which psychophysical studies have shown humans can do. Thus, our data suggest the importance of non-linearity of the early photoreceptor for perception. Additionally, we found that stimulus skewness leads to a subtle change in photoreceptor kinetics. For negatively skewed stimuli, the impulse response functions of the cone peak later than for positively skewed stimuli. However, stimulus skewness does not affect the overall integration time of the cone. KEY POINTS: Humans can discriminate visual scenes based on skewness, i.e. the relative prevalence of bright and dark patches within a scene. Here, we show that negatively skewed time-series stimuli induce larger responses in goldfish cone photoreceptors than comparable positively skewed stimuli. This response asymmetry originates from within the phototransduction cascade, where gain is higher for strong negative contrasts (dark patches) than for strong positive contrasts (bright patches). Unlike the implicit assumption often contained within models of downstream visual neurons, our data show that cone photoreceptors do not simply relay linearly filtered versions of visual stimuli to downstream circuitry, but that they also emphasize specific stimulus features. Given that the phototransduction cascade properties among vertebrate retinas are mostly universal, our data imply that the skew discrimination by human subjects reported in psychophysical studies might stem from the asymmetric gain function of the phototransduction cascade.

Rapid visual adaptation persists across saccades

Neurons in the visual cortex quickly adapt to constant input, which should lead to perceptual fading within few tens of milliseconds. However, perceptual fading is rarely observed in everyday perception, possibly because eye movements refresh retinal input. Recently, it has been suggested that amplitudes of large saccadic eye movements are scaled to maximally decorrelate presaccadic and postsaccadic inputs and thus to annul perceptual fading. However, this argument builds on the assumption that adaptation within naturally brief fixation durations is strong enough to survive any visually disruptive saccade and affect perception. We tested this assumption by measuring the effect of luminance adaptation on postsaccadic contrast perception. We found that postsaccadic contrast perception was affected by presaccadic luminance adaptation during brief periods of fixation. This adaptation effect emerges within 100 milliseconds and persists over seconds. These results indicate that adaptation during natural fixation periods can affect perception even after visually disruptive saccades.

Impact of Photoreceptor Loss on Retinal Circuitry

Our sense of sight relies on photoreceptors, which transduce photons into the nervous system's electrochemical interpretation of the visual world. These precious photoreceptors can be disrupted by disease, injury, and aging. Once photoreceptors start to die, but before blindness occurs, the remaining retinal circuitry can withstand, mask, or exacerbate the photoreceptor deficit and potentially be receptive to newfound therapies for vision restoration. To maximize the retina's receptivity to therapy, one must understand the conditions that influence the state of the remaining retina. In this review, we provide an overview of the retina's structure and function in health and disease. We analyze a collection of observations on photoreceptor disruption and generate a predictive model to identify parameters that influence the retina's response. Finally, we speculate on whether the retina, with its remarkable capacity to function over light levels spanning nine orders of magnitude, uses these same adaptational mechanisms to withstand and perhaps mask photoreceptor loss.

Natural environment statistics in the upper and lower visual field are reflected in mouse retinal specializations

Pressures for survival make sensory circuits adapted to a species' natural habitat and its behavioral challenges. Thus, to advance our understanding of the visual system, it is essential to consider an animal's specific visual environment by capturing natural scenes, characterizing their statistical regularities, and using them to probe visual computations. Mice, a prominent visual system model, have salient visual specializations, being dichromatic with enhanced sensitivity to green and UV in the dorsal and ventral retina, respectively. However, the characteristics of their visual environment that likely have driven these adaptations are rarely considered. Here, we built a UV-green-sensitive camera to record footage from mouse habitats. This footage is publicly available as a resource for mouse vision research. We found chromatic contrast to greatly diverge in the upper, but not the lower, visual field. Moreover, training a convolutional autoencoder on upper, but not lower, visual field scenes was sufficient for the emergence of color-opponent filters, suggesting that this environmental difference might have driven superior chromatic opponency in the ventral mouse retina, supporting color discrimination in the upper visual field. Furthermore, the upper visual field was biased toward dark UV contrasts, paralleled by more light-offset-sensitive ganglion cells in the ventral retina. Finally, footage recorded at twilight suggests that UV promotes aerial predator detection. Our findings support that natural scene statistics shaped early visual processing in evolution.

Different roles of subcortical inputs in V1 responses to luminance and contrast

Cells in the primary visual cortex (V1) generally respond weakly to large uniform luminance stimuli. Only a subset of V1 cells is thought to encode uniform luminance information. In natural scenes, local luminance is an important feature for defining an object that varies and coexists with local spatial contrast. However, the strategies used by V1 cells to encode local mean luminance for spatial contrast stimuli remain largely unclear. Here, using extracellular recordings in anesthetized cats, we investigated the responses of V1 cells by comparing with those of retinal ganglion (RG) cells and lateral geniculate nucleus (LGN) cells to simultaneous and rapid changes in luminance and spatial contrast. Almost all V1 cells exhibited a strong monotonic increasing luminance tuning when they were exposed to high spatial contrast. Thus, V1 cells encode the luminance carried by spatial contrast stimuli with the monotonically increasing response function. Moreover, high contrast decreased luminance tuning of OFF cells but increased that of in ON cells in RG and LGN. The luminance and contrast tunings of LGN ON cells were highly separable as V1 cells, whereas those of LGN OFF cells were lowly separable. These asymmetrical effects of spatial contrast on ON/OFF channels might underlie the robust ability of V1 cells to perform luminance tuning when exposed to spatial contrast stimuli.

A Fast Preprocessing Method for Micro-Expression Spotting via Perceptual Detection of Frozen Frames

This paper presents a preliminary study concerning a fast preprocessing method for facial microexpression (ME) spotting in video sequences. The rationale is to detect frames containing frozen expressions as a quick warning for the presence of MEs. In fact, those frames can either precede or follow (or both) MEs according to ME type and the subject's reaction. To that end, inspired by the Adelson-Bergen motion energy model and the instinctive nature of the preattentive vision, global visual perception-based features were employed for the detection of frozen frames. Preliminary results achieved on both controlled and uncontrolled videos confirmed that the proposed method is able to correctly detect frozen frames and those revealing the presence of nearby MEs-independently of ME kind and facial region. This property can then contribute to speeding up and simplifying the ME spotting process, especially during long video acquisitions.

Image luminance changes contrast sensitivity in visual cortex

Accurate measures of contrast sensitivity are important for evaluating visual disease progression and for navigation safety. Previous measures suggested that cortical contrast sensitivity was constant across widely different luminance ranges experienced indoors and outdoors. Against this notion, here, we show that luminance range changes contrast sensitivity in both cat and human cortex, and the changes are different for dark and light stimuli. As luminance range increases, contrast sensitivity increases more within cortical pathways signaling lights than those signaling darks. Conversely, when the luminance range is constant, light-dark differences in contrast sensitivity remain relatively constant even if background luminance changes. We show that a Naka-Rushton function modified to include luminance range and light-dark polarity accurately replicates both the statistics of light-dark features in natural scenes and the cortical responses to multiple combinations of contrast and luminance. We conclude that differences in light-dark contrast increase with luminance range and are largest in bright environments.

Preprocessing in digital lensless holographic microscopy for intensity reconstructions with enhanced contrast

In this work, a numerical method to enhance the contrast of intensity hologram reconstructions of digital lensless holographic microscopy (DLHM) is presented. The method manipulates the in-line hologram and reference images through mathematical operations between them; additionally, a sharpening operation, functionalized in terms of the parameters of the recording setup, is applied to the said images. The preprocessing of the recorded images produces a modified in-line hologram and a reference wave image from which an intensity reconstruction with a 25% improvement of its contrast, with respect to the conventional reconstruction procedure, is achieved. The method is illustrated with intensity reconstructions of a hologram of a monolayer of polystyrene spheres 1.09 µm in diameter. Finally, the preprocessing method is validated with a modeled hologram, successfully applied to holograms of the section of the head a Drosophila melanogaster fly and its results are contrasted with those obtained via bright-field microscopy.

Flowers in the Attic: Lateralization of the detection of meaning in visual noise

The brain is a slave to sense; we see and hear things that are not there and engage in ongoing correction of these illusory experiences, commonly termed pareidolia. The current study investigates whether the predisposition to see meaning in noise is lateralized to one hemisphere or the other and how this predisposition to visual false-alarms is related to personality. Stimuli consisted of images of faces or flowers embedded in pink (1/f) noise generated through a novel process and presented in a divided-field paradigm. Right-handed undergraduates participated in a forced-choice signal-detection task where they determined whether a face or flower signal was present in a single-interval trial. Experiment 1 involved an equal ratio of signal-to-noise trials; experiment 2 provided more potential for illusionary perception with 25% signal and 75% noise trials. There was no asymmetry in the ability to discriminate signal from noise trials (measured using d') for either faces and flowers, although the response criterion (c) suggested a stronger predisposition to visual false alarms in the right visual field, and this was negatively correlated to the unusual experiences dimension of schizotypy. Counter to expectations, changing the signal-image to noise-image proportion in Experiment 2 did not change the number of false alarms for either faces and flowers, although a stronger bias was seen to the right visual field; sensitivity remained the same in both hemifields but there was a moderate positive correlation between cognitive disorganization and the bias (c) for "flower" judgements. Overall, these results were consistent with a rapid evidence-accumulation process of the kind described by a diffusion decision model mediating the task lateralized to the left-hemisphere.

Evaluation of non-traditional visualization methods to detect surface attachment of biofilms

In food safety and food quality, biofilm research is of great importance for mitigating food-borne pathogens in food processing environments. To supplement the traditional staining techniques for biofilm characterization, we introduce several non-traditional imaging methods for detecting biofilm attachment to the solid-liquid and air-liquid interfaces. For strains of Pseudomonas aeruginosa (the positive control), Acinetobacter baumanii, Listeria monocytogenes and Salmonella enterica, the traditional crystal violet assay showed evidence of biofilm attachment to the well plate base as well as inferred the presence of an air-liquid biofilm attached on the upper well walls where the meniscus was present. However, air-liquid biofilms and solid-surface-attached biofilms were not detected for all of these strains using the non-traditional imaging methods. For L. monocytogenes, we were unable to detect biofilms at a particle-laden, air-liquid interface as evidenced through microscopy, which contradicts the meniscus staining test and suggests that the coffee-ring effect may lead to false positives when using meniscus staining. Furthermore, when L. monocytogenes was cultivated in a pendant droplet in air, only microbial sediment at the droplet apex was observed without any apparent bacterial colonization of the droplet surface. All other strains showed clear evidence of air-liquid biofilms at the air-liquid interface of a pendant droplet. To non-invasively detect if and when air-liquid pellicles form in a well plate, we also present a novel in situ reflection assay that demonstrates the capacity to do this quantitatively.

Functional Specialization of ON and OFF Cortical Pathways for Global-Slow and Local-Fast Vision

Visual information is processed in the cortex by ON and OFF pathways that respond to light and dark stimuli. Responses to darks are stronger, faster, and driven by a larger number of cortical neurons than responses to lights. Here, we demonstrate that these light-dark cortical asymmetries reflect a functional specialization of ON and OFF pathways for different stimulus properties. We show that large long-lasting stimuli drive stronger cortical responses when they are light, whereas small fast stimuli drive stronger cortical responses when they are dark. Moreover, we show that these light-dark asymmetries are preserved under a wide variety of luminance conditions that range from photopic to low mesopic light. Our results suggest that ON and OFF pathways extract different spatiotemporal information from visual scenes, making OFF local-fast signals better suited to maximize visual acuity and ON global-slow signals better suited to guide the eye movements needed for retinal image stabilization.

MARVisT: Authoring Glyph-Based Visualization in Mobile Augmented Reality

Recent advances in mobile augmented reality (AR) techniques have shed new light on personal visualization for their advantages of fitting visualization within personal routines, situating visualization in a real-world context, and arousing users' interests. However, enabling non-experts to create data visualization in mobile AR environments is challenging given the lack of tools that allow in-situ design while supporting the binding of data to AR content. Most existing AR authoring tools require working on personal computers or manually creating each virtual object and modifying its visual attributes. We systematically study this issue by identifying the specificity of AR glyph-based visualization authoring tool and distill four design considerations. Following these design considerations, we design and implement MARVisT, a mobile authoring tool that leverages information from reality to assist non-experts in addressing relationships between data and virtual glyphs, real objects and virtual glyphs, and real objects and data. With MARVisT, users without visualization expertise can bind data to real-world objects to create expressive AR glyph-based visualizations rapidly and effortlessly, reshaping the representation of the real world with data. We use several examples to demonstrate the expressiveness of MARVisT. A user study with non-experts is also conducted to evaluate the authoring experience of MARVisT.

Manipulating the structure of natural scenes using wavelets to study the functional architecture of perceptual hierarchies in the brain

Functional neuroimaging experiments that employ naturalistic stimuli (natural scenes, films, spoken narratives) provide insights into cognitive function "in the wild". Natural stimuli typically possess crowded, spectrally dense, dynamic, and multimodal properties within a rich multiscale structure. However, when using natural stimuli, various challenges exist for creating parametric manipulations with tight experimental control. Here, we revisit the typical spectral composition and statistical dependences of natural scenes, which distinguish them from abstract stimuli. We then demonstrate how to selectively degrade subtle statistical dependences within specific spatial scales using the wavelet transform. Such manipulations leave basic features of the stimuli, such as luminance and contrast, intact. Using functional neuroimaging of human participants viewing degraded natural images, we demonstrate that cortical responses at different levels of the visual hierarchy are differentially sensitive to subtle statistical dependences in natural images. This demonstration supports the notion that perceptual systems in the brain are optimally tuned to the complex statistical properties of the natural world. The code to undertake these stimulus manipulations, and their natural extension to dynamic natural scenes (films), is freely available.

THz imaging technique for nondestructive analysis of debonding defects in ceramic matrix composites based on multiple echoes and feature fusion

We propose a THz nondestructive analysis method based on multiple echoes and feature fusion. Conventionally, it is difficult to identify the debonding defects of the glue layer (II) due to the thin adhesive layer. To this end, a THz propagation model is established, and a quantitative method for determining the thickness of debonding defects based on multiple echoes is presented. The measurement error for preset defect thickness of 500 µm was 4%. Further, for determining the area of debonding defects, a feature fusion imaging algorithm is proposed to realize the lateral recognition of defects and quantitative analysis is used to improve the recognition ability of defects.

Lightness induction enhancements and limitations at low frequency modulations across a variety of stimulus contexts

Lightness illusions are often studied under static viewing conditions with figures varying in geometric design, containing different types of perceptual grouping and figure-ground cues. A few studies have explored the perception of lightness induction while modulating lightness illusions continuously in time, where changes in perceived lightness are often linked to the temporal modulation frequency, up to around 2–4 Hz. These findings support the concept of a cut-off frequency for lightness induction. However, another critical change (enhancement) in the magnitude of perceived lightness during slower temporal modulation conditions has not been addressed in previous temporal modulation studies. Moreover, it remains unclear whether this critical change applies to a variety of lightness illusion stimuli, and the degree to which different stimulus configurations can demonstrate enhanced lightness induction in low modulation frequencies. Therefore, we measured lightness induction strength by having participants cancel out any perceived modulation in lightness detected over time within a central target region, while the surrounding context, which ultimately drives the lightness illusion, was viewed in a static state or modulated continuously in time over a low frequency range (0.25–2 Hz). In general, lightness induction decreased as temporal modulation frequency was increased, with the strongest perceived lightness induction occurring at lower modulation frequencies for visual illusions with strong grouping and figure-ground cues. When compared to static viewing conditions, we found that slow continuous surround modulation induces a strong and significant increase in perceived lightness for multiple types of lightness induction stimuli. Stimuli with perceptually ambiguous grouping and figure-ground cues showed weaker effects of slow modulation lightness enhancement. Our results demonstrate that, in addition to the existence of a cut-off frequency, an additional critical temporal modulation frequency of lightness induction exists (0.25–0.5 Hz), which instead maximally enhances lightness induction and seems to be contingent upon the prevalence of figure-ground and grouping organization.

Luminance potentiates human visuocortical responses

Our visual system is tasked with transforming variations in light within our environment into a coherent percept, typically described using properties such as luminance and contrast. Models of vision often downplay the importance of luminance in shaping cortical responses, instead prioritizing representations that do not covary with overall luminance (i.e., contrast), and yet visuocortical response properties that may reflect luminance encoding remain poorly understood. In this study, we examined whether well-established visuocortical response properties may also reflect luminance encoding, challenging the idea that luminance information itself plays no significant role in supporting visual perception. To do so, we measured functional activity in human visual cortex when presenting stimuli varying in contrast and mean luminance, and found that luminance response functions are strongly contrast dependent between 50 and 250 cd/m², confirmed with a subsequent experiment. High-contrast stimuli produced linearly increasing responses as luminance increased logarithmically for all early visual areas, whereas low-contrast stimuli produced either flat (V1) or assorted positive linear (V2 and V3) response profiles. These results reveal that the mean luminance information of a visual signal persists within visuocortical representations, potentially reflecting an inherent imbalance of excitatory and inhibitory components that can be either contrast dependent (V1 and V2) or contrast invariant (V3). The role of luminance should be considered when the aim is to drive potent visually evoked responses and when activity is compared across studies. More broadly, overall luminance should be weighed heavily as a core feature of the visual system and should play a significant role in cortical models of vision.NEW & NOTEWORTHY This neuroimaging study investigates the influence of overall luminance on population activity in human visual cortex. We discovered that the response to a particular stimulus contrast level is reliant, in part, on the mean luminance of a signal, revealing that the mean luminance information of our environment is represented within the visual cortex. The results challenge a long-standing misconception about the role of luminance information in the processing of visual information at the cortical level.

Luminance Information Is Required for the Accurate Estimation of Contrast in Rapidly Changing Visual Contexts

Visual perception scales with changes in the visual stimulus, or contrast, irrespective of background illumination. However, visual perception is challenged when adaptation is not fast enough to deal with sudden declines in overall illumination, for example, when gaze follows a moving object from bright sunlight into a shaded area. Here, we show that the visual system of the fly employs a solution by propagating a corrective luminance-sensitive signal. We use in vivo 2-photon imaging and behavioral analyses to demonstrate that distinct OFF-pathway inputs encode contrast and luminance. Predictions of contrast-sensitive neuronal responses show that contrast information alone cannot explain behavioral responses in sudden dim light. The luminance-sensitive pathway via the L3 neuron is required for visual processing in such rapidly changing light conditions, ensuring contrast constancy when pure contrast sensitivity underestimates a stimulus. Thus, retaining a peripheral feature, luminance, in visual processing is required for robust behavioral responses.

Image statistics of the environment surrounding freely behaving hoverflies

Natural scenes are not as random as they might appear, but are constrained in both space and time. The 2-dimensional spatial constraints can be described by quantifying the image statistics of photographs. Human observers perceive images with naturalistic image statistics as more pleasant to view, and both fly and vertebrate peripheral and higher order visual neurons are tuned to naturalistic image statistics. However, for a given animal, what is natural differs depending on the behavior, and even if we have a broad understanding of image statistics, we know less about the scenes relevant for particular behaviors. To mitigate this, we here investigate the image statistics surrounding Episyrphus balteatus hoverflies, where the males hover in sun shafts created by surrounding trees, producing a rich and dense background texture and also intricate shadow patterns on the ground. We quantified the image statistics of photographs of the ground and the surrounding panorama, as the ventral and lateral visual field is particularly important for visual flight control, and found differences in spatial statistics in photos where the hoverflies were hovering compared to where they were flying. Our results can, in the future, be used to create more naturalistic stimuli for experimenter-controlled experiments in the laboratory.

Both a Gauge and a Filter: Cognitive Modulations of Pupil Size

Over 50 years of research have established that cognitive processes influence pupil size. This has led to the widespread use of pupil size as a peripheral measure of cortical processing in psychology and neuroscience. However, the function of cortical control over the pupil remains poorly understood. Why does visual attention change the pupil light reflex? Why do mental effort and surprise cause pupil dilation? Here, we consider these functional questions as we review and synthesize two literatures on cognitive effects on the pupil: how cognition affects pupil light response and how cognition affects pupil size under constant luminance. We propose that cognition may have co-opted control of the pupil in order to filter incoming visual information to optimize it for particular goals. This could complement other cortical mechanisms through which cognition shapes visual perception.

Visual evoked cortical potential elicited by pseudoisochromatic stimulus

PURPOSE

Visual evoked cortical potentials (VECPs) are useful for investigating the mechanisms and dysfunctions of color vision. Chromatic sinusoidal gratings are generally used to elicit VECPs, but they require long psychophysical measurements to match the perceptual luminance between their stripes. An alternative method is to use pseudoisochromatic stimuli, which makes use of luminance noise to mask luminance clues and force the target perception to be dependent on chromatic contrast. In this study, we compared VECPs generated by sinusoidal gratings and pseudoisochromatic gratings. Contrary to chromatic sinusoidal gratings, pseudoisochromatic stimuli do not require the use of previous methods to find the equiluminance of the stimulus.

METHODS

Normal trichromats were recruited to be tested with red-green chromatic sinusoidal gratings and pseudoisochromatic gratings presented by pattern onset-offset and pattern reversal modes in five spatial frequencies. In addition, we also tested four different chromatic contrast pairs in pattern onset-offset mode presentation in five trichromats and one colorblind subject (deuteranope).

RESULTS

Pattern onset-offset VECPs elicited by sinusoidal gratings had a larger amplitude than those obtained with pseudoisochromatic stimuli, whereas pattern reversal VECPs elicited by pseudoisochromatic gratings had similar amplitudes compared to those elicited by sinusoidal gratings. We found no difference between the VECP amplitudes elicited by sinusoidal and pseudoisochromatic gratings containing different chromatic contrast. Color-blind subjects displayed absent or small responses to the stimuli.

CONCLUSION

Pseudoisochromatic stimulus can be an alternative stimulus to generate VECPs dominated by the chromatic mechanism.

Stimulus- and goal-oriented frameworks for understanding natural vision

Our knowledge of sensory processing has advanced dramatically in the last few decades, but this understanding remains far from complete, especially for stimuli with the large dynamic range and strong temporal and spatial correlations characteristic of natural visual inputs. Here we describe some of the issues that make understanding the encoding of natural images a challenge. We highlight two broad strategies for approaching this problem: a stimulus-oriented framework and a goal-oriented one. Different contexts can call for one framework or the other. Looking forward, recent advances, particularly those based in machine learning, show promise in borrowing key strengths of both frameworks and by doing so illuminating a path to a more comprehensive understanding of the encoding of natural stimuli.

The Perpetual Diamond: Contrast Reversals Along Thin Edges Create the Appearance of Motion in Objects

The Perpetual Diamond produces motion continuously and unambiguously in one direction despite never physically changing location. The phenomenon consists of a steady, mid-luminance diamond bordered by four thin edge strips and a surrounding background field. The direction of motion is determined by the relative phases of the luminance modulation between the edge strips and the background. Because the motion is generated entirely by changing contrast signals between the edge strips and background, the stimulus is a valuable tool for tests of spatial contrast, temporal contrast, contrast gain, and color contrast. We demonstrate that observers see motion even when the edge strips subtend only seconds of arc on the retina (which is less than the frequently reported 10 minutes of arc) and that perceived motion is due entirely to changes in the difference in contrast phase modulation, independent from the luminance phase.

The dynamic receptive fields of retinal ganglion cells

Retinal ganglion cells (RGCs) were one of the first classes of sensory neurons to be described in terms of a receptive field (RF). Over the last six decades, our understanding of the diversity of RGC types and the nuances of their response properties has grown exponentially. We will review the current understanding of RGC RFs mostly from studies in mammals, but including work from other vertebrates as well. We will argue for a new paradigm that embraces the fluidity of RGC RFs with an eye toward the neuroethology of vision. Specifically, we will focus on (1) different methods for measuring RGC RFs, (2) RF models, (3) feature selectivity and the distinction between fluid and stable RF properties, and (4) ideas about the future of understanding RGC RFs.

Natural image and receptive field statistics predict saccade sizes

Humans and other primates sample the visual environment using saccadic eye movements that shift a high-resolution fovea toward regions of interest to create a clear perception of a scene across fixations. Many mammals, however, like mice, lack a fovea, which raises the question of why they make saccades. Here we describe and test the hypothesis that saccades are matched to natural scene statistics and to the receptive field sizes and adaptive properties of neural populations. Specifically, we determined the minimum amplitude of saccades in natural scenes necessary to provide uncorrelated inputs to model neural populations. This analysis predicts the distributions of observed saccade sizes during passive viewing for nonhuman primates, cats, and mice. Furthermore, disrupting the development of receptive field properties by monocular deprivation changed saccade sizes consistent with this hypothesis. Therefore, natural-scene statistics and the neural representation of natural images appear to be critical factors guiding saccadic eye movements.

No Reference Quality Assessment for Screen Content Images With Both Local and Global Feature Representation

In this paper, we propose a novel no reference quality assessment method by incorporating statistical luminance and texture features (NRLT) for screen content images (SCIs) with both local and global feature representation. The proposed method is designed inspired by the perceptual property of the human visual system (HVS) that the HVS is sensitive to luminance change and texture information for image perception. In the proposed method, we first calculate the luminance map through the local normalization, which is further used to extract the statistical luminance features in global scope. Second, inspired by existing studies from neuroscience that high-order derivatives can capture image texture, we adopt four filters with different directions to compute gradient maps from the luminance map. These gradient maps are then used to extract the second-order derivatives by local binary pattern. We further extract the texture feature by the histogram of high-order derivatives in global scope. Finally, support vector regression is applied to train the mapping function from quality-aware features to subjective ratings. Experimental results on the public large-scale SCI database show that the proposed NRLT can achieve better performance in predicting the visual quality of SCIs than relevant existing methods, even including some full reference visual quality assessment methods.

Neuronal mechanisms underlying differences in spatial resolution between darks and lights in human vision

Artists and astronomers noticed centuries ago that humans perceive dark features in an image differently from light ones; however, the neuronal mechanisms underlying these dark/light asymmetries remained unknown. Based on computational modeling of neuronal responses, we have previously proposed that such perceptual dark/light asymmetries originate from a luminance/response saturation within the ON retinal pathway. Consistent with this prediction, here we show that stimulus conditions that increase ON luminance/response saturation (e.g., dark backgrounds) or its effect on light stimuli (e.g., optical blur) impair the perceptual discrimination and salience of light targets more than dark targets in human vision. We also show that, in cat visual cortex, the magnitude of the ON luminance/response saturation remains relatively constant under a wide range of luminance conditions that are common indoors, and only shifts away from the lowest luminance contrasts under low mesopic light. Finally, we show that the ON luminance/response saturation affects visual salience mostly when the high spatial frequencies of the image are reduced by poor illumination or optical blur. Because both low luminance and optical blur are risk factors in myopia, our results suggest a possible neuronal mechanism linking myopia progression with the function of the ON visual pathway.