Changes in the intensity-response function of an insect's photoreceptors due to light adaptation

Detecting Small Size and Minimal Thermal Signature Targets in Infrared Imagery Using Biologically Inspired Vision

Thermal infrared imaging provides an effective sensing modality for detecting small moving objects at long range. Typical challenges that limit the efficiency and robustness of the detection performance include sensor noise, minimal target contrast and cluttered backgrounds. These issues become more challenging when the targets are of small physical size and present minimal thermal signatures. In this paper, we experimentally show that a four-stage biologically inspired vision (BIV) model of the flying insect visual system have an excellent ability to overcome these challenges simultaneously. The early two stages of the model suppress spatio-temporal clutter and enhance spatial target contrast while compressing the signal in a computationally manageable bandwidth. The later two stages provide target motion enhancement and sub-pixel motion detection capabilities. To show the superiority of the BIV target detector over existing traditional detection methods, we perform extensive experiments and performance comparisons using high bit-depth, real-world infrared image sequences of small size and minimal thermal signature targets at long ranges. Our results show that the BIV target detector significantly outperformed 10 conventional spatial-only and spatiotemporal methods for infrared small target detection. The BIV target detector resulted in over 25 dB improvement in the median signal-to-clutter-ratio over the raw input and achieved 43% better detection rate than the best performing existing method.

Sparse identification of contrast gain control in the fruit fly photoreceptor and amacrine cell layer

The fruit fly's natural visual environment is often characterized by light intensities ranging across several orders of magnitude and by rapidly varying contrast across space and time. Fruit fly photoreceptors robustly transduce and, in conjunction with amacrine cells, process visual scenes and provide the resulting signal to downstream targets. Here, we model the first step of visual processing in the photoreceptor-amacrine cell layer. We propose a novel divisive normalization processor (DNP) for modeling the computation taking place in the photoreceptor-amacrine cell layer. The DNP explicitly models the photoreceptor feedforward and temporal feedback processing paths and the spatio-temporal feedback path of the amacrine cells. We then formally characterize the contrast gain control of the DNP and provide sparse identification algorithms that can efficiently identify each the feedforward and feedback DNP components. The algorithms presented here are the first demonstration of tractable and robust identification of the components of a divisive normalization processor. The sparse identification algorithms can be readily employed in experimental settings, and their effectiveness is demonstrated with several examples.

Microsaccadic sampling of moving image information provides Drosophila hyperacute vision

Small fly eyes should not see fine image details. Because flies exhibit saccadic visual behaviors and their compound eyes have relatively few ommatidia (sampling points), their photoreceptors would be expected to generate blurry and coarse retinal images of the world. Here we demonstrate that Drosophila see the world far better than predicted from the classic theories. By using electrophysiological, optical and behavioral assays, we found that R1-R6 photoreceptors’ encoding capacity in time is maximized to fast high-contrast bursts, which resemble their light input during saccadic behaviors. Whilst over space, R1-R6s resolve moving objects at saccadic speeds beyond the predicted motion-blur-limit. Our results show how refractory phototransduction and rapid photomechanical photoreceptor contractions jointly sharpen retinal images of moving objects in space-time, enabling hyperacute vision, and explain how such microsaccadic information sampling exceeds the compound eyes’ optical limits. These discoveries elucidate how acuity depends upon photoreceptor function and eye movements.

Fruit flies have five eyes: two large compound eyes which support vision, plus three smaller single lens eyes which are used for navigation. Each compound eye monitors 180° of space and consists of roughly 750 units, each containing eight light-sensitive cells called photoreceptors. This relatively wide spacing of photoreceptors is thought to limit the sharpness, or acuity, of vision in fruit flies. The area of the human retina (the light-sensitive surface at back of our eyes) that generates our sharpest vision contains photoreceptors that are 500 times more densely packed.

Despite their differing designs, human and fruit fly eyes work via the same general principles. If we, or a fruit fly, were to hold our gaze completely steady, the world would gradually fade from view as the eye adapted to the unchanging visual stimulus. To ensure this does not happen, animals continuously make rapid, automatic eye movements called microsaccades. These refresh the image on the retina and prevent it from fading. Yet it is not known why do they not also cause blurred vision.

Standard accounts of vision assume that the retina and the brain perform most of the information processing required, with photoreceptors simply detecting how much light enters the eye. However, Juusola, Dau, Song et al. now challenge this idea by showing that photoreceptors are specially adapted to detect the fluctuating patterns of light that enter the eye as a result of microsaccades. Moreover, fruit fly eyes resolve small moving objects far better than would be predicted based on the spacing of their photoreceptors.

The discovery that photoreceptors are well adapted to deal with eye movements changes our understanding of insect vision. The findings also disprove the 100-year-old dogma that the spacing of photoreceptors limits the sharpness of vision in compound eyes. Further studies are required to determine whether photoreceptors in the retinas of other animals, including humans, have similar properties.

Developing photoreceptor-based models of visual attraction in riverine tsetse, for use in the engineering of more-attractive polyester fabrics for control devices

Riverine tsetse transmit the parasites that cause the most prevalent form of human African trypanosomiasis, Gambian HAT. In response to the imperative for cheap and efficient tsetse control, insecticide-treated 'tiny targets' have been developed through refinement of tsetse attractants based on blue fabric panels. However, modern blue polyesters used for this purpose attract many less tsetse than traditional phthalogen blue cottons. Therefore, colour engineering polyesters for improved attractiveness has great potential for tiny target development. Because flies have markedly different photoreceptor spectral sensitivities from humans, and the responses of these photoreceptors provide the inputs to their visually guided behaviours, it is essential that polyester colour engineering be guided by fly photoreceptor-based explanations of tsetse attraction. To this end, tsetse attraction to differently coloured fabrics was recently modelled using the calculated excitations elicited in a generic set of fly photoreceptors as predictors. However, electrophysiological data from tsetse indicate the potential for modified spectral sensitivities versus the generic pattern, and processing of fly photoreceptor responses within segregated achromatic and chromatic channels has long been hypothesised. Thus, I constructed photoreceptor-based models explaining the attraction of G. f. fuscipes to differently coloured tiny targets recorded in a previously published investigation, under differing assumptions about tsetse spectral sensitivities and organisation of visual processing. Models separating photoreceptor responses into achromatic and chromatic channels explained attraction better than earlier models combining weighted photoreceptor responses in a single mechanism, regardless of the spectral sensitivities assumed. However, common principles for fabric colour engineering were evident across the complete set of models examined, and were consistent with earlier work. Tools for the calculation of fly photoreceptor excitations are available with this paper, and the ways in which these and photoreceptor-based models of attraction can provide colorimetric values for the engineering of more-attractively coloured polyester fabrics are discussed.

Current advances in invertebrate vision: insights from patch-clamp studies of photoreceptors in apposition eyes

Traditional electrophysiological research on invertebrate photoreceptors has been conducted in vivo, using intracellular recordings from intact compound eyes. The only exception used to be Drosophila melanogaster, which was exhaustively studied by both intracellular recording and patch-clamp methods. Recently, several patch-clamp studies have provided new information on the biophysical properties of photoreceptors of diverse insect species, having both apposition and neural superposition eyes, in the contexts of visual ecology, behavior, and ontogenesis. Here, I discuss these and other relevant results, emphasizing differences between fruit flies and other species, between photoreceptors of diurnal and nocturnal insects, properties of distinct functional types of photoreceptors, postembryonic developmental changes, and relationships between voltage-gated potassium channels and visual ecology.

An artificial elementary eye with optic flow detection and compositional properties

We describe a 2 mg artificial elementary eye whose structure and functionality is inspired by compound eye ommatidia. Its optical sensitivity and electronic architecture are sufficient to generate the required signals for the measurement of local optic flow vectors in multiple directions. Multiple elementary eyes can be assembled to create a compound vision system of desired shape and curvature spanning large fields of view. The system configurability is validated with the fabrication of a flexible linear array of artificial elementary eyes capable of extracting optic flow over multiple visual directions.

A colour opponent model that explains tsetse fly attraction to visual baits and can be used to investigate more efficacious bait materials

Palpalis group tsetse flies are the major vectors of human African trypanosomiasis, and visually-attractive targets and traps are important tools for their control. Considerable efforts are underway to optimise these visual baits, and one factor that has been investigated is coloration. Analyses of the link between visual bait coloration and tsetse fly catches have used methods which poorly replicate sensory processing in the fly visual system, but doing so would allow the visual information driving tsetse attraction to these baits to be more fully understood, and the reflectance spectra of candidate visual baits to be more completely analysed. Following methods well established for other species, I reanalyse the numbers of tsetse flies caught at visual baits based upon the calculated photoreceptor excitations elicited by those baits. I do this for large sets of previously published data for Glossina fuscipes fuscipes (Lindh et al. (2012). PLoS Negl Trop Dis 6: e1661), G. palpalis palpalis (Green (1988). Bull Ent Res 78: 591), and G. pallidipes (Green and Flint (1986). Bull Ent Res 76: 409). Tsetse attraction to visual baits in these studies can be explained by a colour opponent mechanism to which the UV-blue photoreceptor R7y contributes positively, and both the green-yellow photoreceptor R8y, and the low-wavelength UV photoreceptor R7p, contribute negatively. A tool for calculating fly photoreceptor excitations is made available with this paper, and this will facilitate a complete and biologically authentic description of visual bait reflectance spectra that can be employed in the search for more efficacious visual baits, or the analysis of future studies of tsetse fly attraction.

Opsin expression, physiological characterization and identification of photoreceptor cells in the dorsal rim area and main retina of the desert locust, Schistocerca gregaria

For compass orientation many insects rely on the pattern of sky polarization but some species also exploit the sky chromatic contrast. Desert locusts, Schistocerca gregaria, detect polarized light through a specialized dorsal rim area (DRA) in their compound eye. To better understand retinal mechanisms underlying visual navigation, we compared opsin expression, spectral and polarization sensitivities and response-stimulus intensity functions in the DRA and main retina of the locust. In addition to previously characterized opsins of long-wavelength-absorbing (Lo1) and blue-absorbing visual pigments (Lo2), we identified an opsin of a UV-absorbing visual pigment (LoUV). DRA photoreceptors exclusively expressed Lo2, had peak spectral sensitivities at 441 nm and showed high polarization sensitivity (PS 1.3-31.7). In contrast, ommatidia in the main eye coexpressed Lo1 and Lo2 in five photoreceptors, expressed Lo1 in two proximal photoreceptors, and Lo2 or LoUV in one distal photoreceptor. Correspondingly, we found broadband blue- and green-peaking spectral sensitivities in the main eye and one narrowly tuned UV peaking receptor. Polarization sensitivity in the main retina was low (PS 1.3-3.8). V-log I functions in the DRA were steeper than in the main retina supporting a role in polarization vision. Desert locusts occur as two morphs, a day-active gregarious and a night-active solitarious form. In solitarious locusts sensitivities in the main retina were generally shifted to longer wavelengths, particularly in ventral eye regions, supporting a nocturnal life style at low light levels. The data support the role of the DRA in polarization vision and suggest trichromatic colour vision in the desert locust.

Macroscopic characteristics of the praying mantis electroretinogram

We described the macroscopic characteristics of the praying mantis ERG in three species, Tenodera aridifolia sinensis, Sphodromantis lineola, and Popa spurca. In all cases, when elicited by square wave light pulses longer than 400 ms, light adapted (LA) ERGs consisted of four component waveforms: a cornea negative transient and sustained ON, a cornea negative transient OFF, and a cornea positive sustained OFF. The former two ON, and the latter OFF components were attributed to photoreceptor depolarization and repolarization, respectively. Metabolic stress via CO2 induced anoxia selectively eliminated the transient OFF (independent of its effect on the other components) suggesting the transient OFF represents activity of the lamina interneurons on which the photoreceptors synapse. Dark adapted (DA) ERGs differed from LA ERGs in that the sustained ON and OFF amplitudes were larger, and the transient ON and OFF components were absent. Increased stimulus durations increased the amplitudes and derivatives of, and decreased the latencies to the maximum amplitudes of the OFF components. Increasing stimulus intensity increased the amplitude of the sustained ON and OFF components, but not the transient OFF. These results suggest that the mantis' visual system displays increased contrast coding efficiency with increased light adaptation, and that there are differences in gain between photoreceptor and lamina interneuron responses. Finally, responses to luminance decrements as brief a 1 ms were evident in LA recordings, and were resolved at frequencies up to 60 Hz.

Miniature curved artificial compound eyes

In most animal species, vision is mediated by compound eyes, which offer lower resolution than vertebrate single-lens eyes, but significantly larger fields of view with negligible distortion and spherical aberration, as well as high temporal resolution in a tiny package. Compound eyes are ideally suited for fast panoramic motion perception. Engineering a miniature artificial compound eye is challenging because it requires accurate alignment of photoreceptive and optical components on a curved surface. Here, we describe a unique design method for biomimetic compound eyes featuring a panoramic, undistorted field of view in a very thin package. The design consists of three planar layers of separately produced arrays, namely, a microlens array, a neuromorphic photodetector array, and a flexible printed circuit board that are stacked, cut, and curved to produce a mechanically flexible imager. Following this method, we have prototyped and characterized an artificial compound eye bearing a hemispherical field of view with embedded and programmable low-power signal processing, high temporal resolution, and local adaptation to illumination. The prototyped artificial compound eye possesses several characteristics similar to the eye of the fruit fly Drosophila and other arthropod species. This design method opens up additional vistas for a broad range of applications in which wide field motion detection is at a premium, such as collision-free navigation of terrestrial and aerospace vehicles, and for the experimental testing of insect vision theories.

A Biologically Inspired Smart Camera for Use in Surveillance Applications

Biological vision systems are capable of discerning detail as well as detecting objects and motion in a wide range of highly variable lighting conditions that proves challenging to traditional cameras. In this paper, the authors describe the real-time implementation of a biological vision model using a high dynamic range video camera and a General Purpose Graphics Processing Unit. The effectiveness of this implementation is demonstrated in two surveillance applications: dynamic equalization of contrast for improved recognition of scene detail and the use of biologically-inspired motion processing for the detection of small or distant moving objects in a complex scene. A system based on this prototype could improve surveillance capability in any number of difficult situations.

Adaptation accentuates responses of fly motion-sensitive visual neurons to sudden stimulus changes

Adaptation in sensory and neuronal systems usually leads to reduced responses to persistent or frequently presented stimuli. In contrast to simple fatigue, adapted neurons often retain their ability to encode changes in stimulus intensity and to respond when novel stimuli appear. We investigated how the level of adaptation of a fly visual motion-sensitive neuron affects its responses to discontinuities in the stimulus, i.e. sudden brief changes in one of the stimulus parameters (velocity, contrast, grating orientation and spatial frequency). Although the neuron's overall response decreased gradually during ongoing motion stimulation, the response transients elicited by stimulus discontinuities were preserved or even enhanced with adaptation. Moreover, the enhanced sensitivity to velocity changes by adaptation was not restricted to a certain velocity range, but was present regardless of whether the neuron was adapted to a baseline velocity below or above its steady-state velocity optimum. Our results suggest that motion adaptation helps motion-sensitive neurons to preserve their sensitivity to novel stimuli even in the presence of strong tonic stimulation, for example during self-motion.

Implementation of an elaborated neuromorphic model of a biological photoreceptor

We describe here an elaborated neuromorphic model based on the photoreceptors of flies and realised in both software simulation and hardware using discrete circuit components. The design of the model is based on optimisations and further elaborations to the mathematical model initially developed by van Hateren and Snippe that has been shown to accurately simulate biological responses in simulations under both steady-state and limited dynamic conditions. The model includes an adaptive time constant, nonlinear adaptive gain control, logarithmic saturation and a nonlinear adaptive frequency response mechanism. It consists of a linear phototransduction stage, a dynamic filter stage, two divisive feedback loops and a static nonlinearity. In order to test the biological accuracy of the model, impulses and step responses were used to test and evaluate the steady-state characteristics of both the biological (fly) and artificial (new neuromorphic model) photoreceptors. These tests showed that the model has faithfully captured most of the essential characteristics of the insect photoreceptor cells. The model showed a decreasing response to impulsive stimuli when the background intensity was increased, indicating that the circuit adapted to background luminance in order to improve the overall operating range and better encode the contrast of the stimulus rather than luminance. The model also showed the same change in its frequency response characteristics as the biological photoreceptors over a luminance range of 70,000 cd/m(2), with the corner frequency of the circuit ranging from 10 to 90 Hz depending on the current state of adaptation. Complex naturalistic experiments have also further proven the robustness of the model to perform in real-world scenario. The model showed great correlation to the biological photoreceptors with an r (2) value exceeding 0.83. Our model could act as an excellent platform for future experiments that could be carried out in scenarios where in vivo intracellular recording from biological photoreceptors would be impractical or impossible, or as a front-end for an artificial imaging system.

Light dependence of calcium and membrane potential measured in blowfly photoreceptors in vivo

Light adaptation in insect photoreceptors is caused by an increase in the cytosolic Ca2+ concentration. To better understand this process, we measured the cytosolic Ca2+ concentration in vivo as a function of adapting light intensity in the white-eyed blowfly mutant chalky. We developed a technique to measure the cytosolic Ca2+ concentration under conditions as natural as possible. The calcium indicator dyes Oregon Green 1, 2, or 5N (Molecular Probes, Inc., Eugene, OR) were iontophoretically injected via an intracellular electrode into a photoreceptor cell in the intact eye; the same electrode was also used to measure the membrane potential. The blue-induced green fluorescence of these dyes could be monitored by making use of the optics of the facet lens and the rhabdomere waveguide. The use of the different Ca2+-sensitive dyes that possess different affinities for Ca2+ allowed the quantitative determination of the cytosolic Ca2+ concentration in the steady state. Determining the cytosolic Ca2+ concentration as a function of the adapting light intensity shows that the Ca2+ concentration is regulated in a graded fashion over the whole dynamic range where a photoreceptor cell can respond to light. When a photoreceptor is adapted to bright light, the cytosolic Ca2+ concentration reaches stable values higher than 10 microM. The data are consistent with the hypothesis that the logarithm of the increase in cytosolic Ca2+ concentration is linear with the logarithm of the light intensity. From the estimated values of the cytosolic Ca2+ concentration, we conclude that the Ca2+-buffering capacity is limited. The percentage of the Ca2+ influx that is buffered gradually decreases with increasing Ca2+ concentrations; at cytosolic Ca2+ concentration levels above 10 microM, buffering becomes minimal.

Stomatopod photoreceptor spectral tuning as an adaptation for colour constancy in water

Where colour is used in communication absolute judgement of signalling spectra is important, and failures of colour constancy may limit performance. Stomatopod crustaceans have unusual eyes in which the midband contains ten or more classes of photoreceptor. For constancy based on receptor adaptation to a fixed background, elementary theory predicts and we confirm by modelling, that stomatopods' narrow-band receptors outperform more broadly tuned receptors. Similar considerations could account for the small spectral separation of receptors in each midband row. Thus, stomatopods seem to trade-off sensitivity and signal-to-noise ratio for increased colour constancy.

Light guides in the dorsal eye of the male mayfly

(i) The dorsal eyes are sensitive to ultraviolet light, which is focused by the corneal lens into crystalline cones in the region where these taper progressively to columns across the clear zone. The action of these columns as light guides can be observed in fixed eyes embedded in polymerized resin. In life the light guide part of the column is surrounded by watery non-cellular haemolymph. (ii) Shadowing the eye surface with a thin wire (three facets wide) while recording from individual receptor units shows that ultraviolet light reaches each receptor by its own facet as in an apposition eye, and not, as in a superposition eye, by a group of many facets. (iii) As shown by the dye Lucifer Yellow injected from a microelectrode, the electrophysiological unit consists of all seven retinula cells in the rhabdom region. Consistent with this tight coupling of retinula cells there is no polarization sensitivity. The peak spectral sensitivity of all single units is at 345-365 nm in the ultraviolet. The acceptance angle is 2.0–2.5°. The sensitivity at the spectral peak to a point source on the optical axis of the unit is poor compared to that in other insects tested with the same equipment. (iv) The acceptance angles (∆ ρ ) in the dorsal eye are at the theoretical minimum for the facet diameter and wavelength from diffraction theory. Ultraviolet vision, therefore, has made possible a reduction in facet size but the interommatidial angle ∆ ϕ is greater than expected from the optimum sampling theory of the diffraction limited compound eye. In fact ∆ ρ ≈ ∆ ϕ ≈ 2°. (v) The dorsal eye is effectively a foveal region with greater sampling density and narrower receptive fields but less overlap of fields than the lateral eye. (vi) The square cones and yellow screening pigment strongly suggest that there is superposition by reflexion of yellow light that spreads between ommatidia across the clear zone. This yellow light might photoreisomerize the visual pigment. Attempts to prove this theory during the recording from single units have so far failed but no better function for the clear zone has been suggested.

Electrical coupling of neuro-ommatidial photoreceptor cells in the blowfly

A new method of microstimulation of the blowfly eye using corneal neutralization was applied to the 6 peripheral photoreceptor cells (R1-R6) connected to one neuro-ommatidium (and thus looking into the same direction), whilst the receptor potential of a dark-adapted photoreceptor cell was recorded by means of an intracellular microelectrode. Stimulation of the photoreceptor cells not impaled elicited responses in the recorded cell of about 20% of the response elicited when stimulating the recorded cell. This is probably caused by gap junctions recently found between the axon terminals of these cells. Stimulation of all 6 cells together yielded responses that were larger and longer than those obtained with stimulation of just the recorded cell, and intensity-response curves that deviated more strongly from linearity. Evidence is presented that the resistance of the axon terminal of the photoreceptor cells quickly drops in response to a light flash, depending on the light intensity. Incorporating the cable properties of the cell body and the axon, the resistance of the gap junctions, and the (adapting) terminal resistance, a theoretical model is presented that explains the measurements well. Finally, it is argued that the gap junctions between the photoreceptor cells may effectively uncouple the synaptic responses of the cells by counteracting the influence of field potentials.