A low-cost hyperspectral scanner for natural imaging and the study of animal colour vision above and under water

Birds multiplex spectral and temporal visual information via retinal On- and Off-channels

In vertebrate vision, early retinal circuits divide incoming visual information into functionally opposite elementary signals: On and Off, transient and sustained, chromatic and achromatic. Together these signals can yield an efficient representation of the scene for transmission to the brain via the optic nerve. However, this long-standing interpretation of retinal function is based on mammals, and it is unclear whether this functional arrangement is common to all vertebrates. Here we show that male poultry chicks use a fundamentally different strategy to communicate information from the eye to the brain. Rather than using functionally opposite pairs of retinal output channels, chicks encode the polarity, timing, and spectral composition of visual stimuli in a highly correlated manner: fast achromatic information is encoded by Off-circuits, and slow chromatic information overwhelmingly by On-circuits. Moreover, most retinal output channels combine On- and Off-circuits to simultaneously encode, or multiplex, both achromatic and chromatic information. Our results from birds conform to evidence from fish, amphibians, and reptiles which retain the full ancestral complement of four spectral types of cone photoreceptors.

Compact and ultracompact spectral imagers: technology and applications in biomedical imaging

Significance

Spectral imaging, which includes hyperspectral and multispectral imaging, can provide images in numerous wavelength bands within and beyond the visible light spectrum. Emerging technologies that enable compact, portable spectral imaging cameras can facilitate new applications in biomedical imaging.

Aim

With this review paper, researchers will (1) understand the technological trends of upcoming spectral cameras, (2) understand new specific applications that portable spectral imaging unlocked, and (3) evaluate proper spectral imaging systems for their specific applications.

Approach

We performed a comprehensive literature review in three databases (Scopus, PubMed, and Web of Science). We included only fully realized systems with definable dimensions. To best accommodate many different definitions of "compact," we included a table of dimensions and weights for systems that met our definition.

Results

There is a wide variety of contributions from industry, academic, and hobbyist spaces. A variety of new engineering approaches, such as Fabry-Perot interferometers, spectrally resolved detector array (mosaic array), microelectro-mechanical systems, 3D printing, light-emitting diodes, and smartphones, were used in the construction of compact spectral imaging cameras. In bioimaging applications, these compact devices were used for in vivo and ex vivo diagnosis and surgical settings.

Conclusions

Compact and ultracompact spectral imagers are the future of spectral imaging systems. Researchers in the bioimaging fields are building systems that are low-cost, fast in acquisition time, and mobile enough to be handheld.

Night skies through animals' eyes-Quantifying night-time visual scenes and light pollution as viewed by animals

A large proportion of animal species enjoy the benefits of being active at night, and have evolved the corresponding optical and neural adaptations to cope with the challenges of low light intensities. However, over the past century electric lighting has introduced direct and indirect light pollution into the full range of terrestrial habitats, changing nocturnal animals' visual worlds dramatically. To understand how these changes affect nocturnal behavior, we here propose an animal-centered analysis method based on environmental imaging. This approach incorporates the sensitivity and acuity limits of individual species, arriving at predictions of photon catch relative to noise thresholds, contrast distributions, and the orientation cues nocturnal species can extract from visual scenes. This analysis relies on just a limited number of visual system parameters known for each species. By accounting for light-adaptation in our analysis, we are able to make more realistic predictions of the information animals can extract from nocturnal visual scenes under different levels of light pollution. With this analysis method, we aim to provide context for the interpretation of behavioral findings, and to allow researchers to generate specific hypotheses for the behavior of nocturnal animals in observed light-polluted scenes.

Ancestral circuits for vertebrate color vision emerge at the first retinal synapse

For color vision, retinal circuits separate information about intensity and wavelength. In vertebrates that use the full complement of four “ancestral” cone types, the nature and implementation of this computation remain poorly understood. Here, we establish the complete circuit architecture of outer retinal circuits underlying color processing in larval zebrafish. We find that the synaptic outputs of red and green cones efficiently rotate the encoding of natural daylight in a principal components analysis–like manner to yield primary achromatic and spectrally opponent axes, respectively. Blue cones are tuned to capture most remaining variance when opposed to green cones, while UV cone present a UV achromatic axis for prey capture. We note that fruitflies use essentially the same strategy. Therefore, rotating color space into primary achromatic and chromatic axes at the eye’s first synapse may thus be a fundamental principle of color vision when using more than two spectrally well-separated photoreceptor types.

Hyperspectral Mapping for the Detection of SARS-CoV-2 Using Nanomolecular Probes with Yoctomole Sensitivity

Efficient monitoring of SARS-CoV-2 outbreak requires the use of a sensitive and rapid diagnostic test. Although SARS-CoV-2 RNA can be detected by RT-qPCR, the molecular-level quantification of the viral load is still challenging, time-consuming, and labor-intensive. Here, we report an ultrasensitive hyperspectral sensor (HyperSENSE) based on hafnium nanoparticles (HfNPs) for specific detection of COVID-19 causative virus, SARS-CoV-2. Density functional theoretical calculations reveal that HfNPs exhibit higher changes in their absorption wavelength and light scattering when bound to their target SARS-CoV-2 RNA sequence relative to the gold nanoparticles. The assay has a turnaround time of a few seconds and has a limit of detection in the yoctomolar range, which is 1 000 000-fold times higher than the currently available COVID-19 tests. We demonstrated in ∼100 COVID-19 clinical samples that the assay is highly sensitive and has a specificity of 100%. We also show that HyperSENSE can rapidly detect other viruses such as influenza A H1N1. The outstanding sensitivity indicates the potential of the current biosensor in detecting the prevailing presymptomatic and asymptomatic COVID-19 cases. Thus, integrating hyperspectral imaging with nanomaterials establishes a diagnostic platform for ultrasensitive detection of COVID-19 that can potentially be applied to any emerging infectious pathogen.

Natural environment statistics in the upper and lower visual field are reflected in mouse retinal specializations

Pressures for survival make sensory circuits adapted to a species' natural habitat and its behavioral challenges. Thus, to advance our understanding of the visual system, it is essential to consider an animal's specific visual environment by capturing natural scenes, characterizing their statistical regularities, and using them to probe visual computations. Mice, a prominent visual system model, have salient visual specializations, being dichromatic with enhanced sensitivity to green and UV in the dorsal and ventral retina, respectively. However, the characteristics of their visual environment that likely have driven these adaptations are rarely considered. Here, we built a UV-green-sensitive camera to record footage from mouse habitats. This footage is publicly available as a resource for mouse vision research. We found chromatic contrast to greatly diverge in the upper, but not the lower, visual field. Moreover, training a convolutional autoencoder on upper, but not lower, visual field scenes was sufficient for the emergence of color-opponent filters, suggesting that this environmental difference might have driven superior chromatic opponency in the ventral mouse retina, supporting color discrimination in the upper visual field. Furthermore, the upper visual field was biased toward dark UV contrasts, paralleled by more light-offset-sensitive ganglion cells in the ventral retina. Finally, footage recorded at twilight suggests that UV promotes aerial predator detection. Our findings support that natural scene statistics shaped early visual processing in evolution.

Looking for Image Statistics: Active Vision With Avatars in a Naturalistic Virtual Environment

The efficient coding hypothesis posits that sensory systems are tuned to the regularities of their natural input. The statistics of natural image databases have been the topic of many studies, which have revealed biases in the distribution of orientations that are related to neural representations as well as behavior in psychophysical tasks. However, commonly used natural image databases contain images taken with a camera with a planar image sensor and limited field of view. Thus, these images do not incorporate the physical properties of the visual system and its active use reflecting body and eye movements. Here, we investigate quantitatively, whether the active use of the visual system influences image statistics across the visual field by simulating visual behaviors in an avatar in a naturalistic virtual environment. Images with a field of view of 120° were generated during exploration of a virtual forest environment both for a human and cat avatar. The physical properties of the visual system were taken into account by projecting the images onto idealized retinas according to models of the eyes' geometrical optics. Crucially, different active gaze behaviors were simulated to obtain image ensembles that allow investigating the consequences of active visual behaviors on the statistics of the input to the visual system. In the central visual field, the statistics of the virtual images matched photographic images regarding their power spectra and a bias in edge orientations toward cardinal directions. At larger eccentricities, the cardinal bias was superimposed with a gradually increasing radial bias. The strength of this effect depends on the active visual behavior and the physical properties of the eye. There were also significant differences between the upper and lower visual field, which became stronger depending on how the environment was actively sampled. Taken together, the results show that quantitatively relating natural image statistics to neural representations and psychophysical behavior requires not only to take the structure of the environment into account, but also the physical properties of the visual system, and its active use in behavior.

Underwater Hyperspectral Imaging Technology and Its Applications for Detecting and Mapping the Seafloor: A Review

Common methods of ocean remote sensing and seafloor surveying are mainly carried out by airborne and spaceborne hyperspectral imagers. However, the water column hinders the propagation of sunlight to deeper areas, thus limiting the scope of observation. As an emerging technology, underwater hyperspectral imaging (UHI) is an extension of hyperspectral imaging technology in air conditions, and is undergoing rapid development for applications in shallow and deep-sea environments. It is a close-range, high-resolution approach for detecting and mapping the seafloor. In this paper, we focus on the concepts of UHI technology, covering imaging systems and the correction methods of eliminating the water column’s influence. The current applications of UHI, such as deep-sea mineral exploration, benthic habitat mapping, and underwater archaeology, are highlighted to show the potential of this technology. This review can provide an introduction and overview for those working in the field and offer a reference for those searching for literature on UHI technology.

Zebrafish Retinal Ganglion Cells Asymmetrically Encode Spectral and Temporal Information across Visual Space

In vertebrate vision, the tetrachromatic larval zebrafish permits non-invasive monitoring and manipulating of neural activity across the nervous system in vivo during ongoing behavior. However, despite a perhaps unparalleled understanding of links between zebrafish brain circuits and visual behaviors, comparatively little is known about what their eyes send to the brain via retinal ganglion cells (RGCs). Major gaps in knowledge include any information on spectral coding and information on potentially critical variations in RGC properties across the retinal surface corresponding with asymmetries in the statistics of natural visual space and behavioral demands. Here, we use in vivo two-photon imaging during hyperspectral visual stimulation as well as photolabeling of RGCs to provide a functional and anatomical census of RGCs in larval zebrafish. We find that RGCs' functional and structural properties differ across the eye and include a notable population of UV-responsive On-sustained RGCs that are only found in the acute zone, likely to support visual prey capture of UV-bright zooplankton. Next, approximately half of RGCs display diverse forms of color opponency, including many that are driven by a pervasive and slow blue-Off system-far in excess of what would be required to satisfy traditional models of color vision. In addition, most information on spectral contrast was intermixed with temporal information. Taken together, our results suggest that zebrafish RGCs send a diverse and highly regionalized time-color code to the brain.