SEMPANet: A Modified Path Aggregation Network with Squeeze-Excitation for Scene Text Detection

Recently, various object detection frameworks have been applied to text detection tasks and have achieved good performance in the final detection. With the further expansion of text detection application scenarios, the research value of text detection topics has gradually increased. Text detection in natural scenes is more challenging for horizontal text based on a quadrilateral detection box and for curved text of any shape. Most networks have a good effect on the balancing of target samples in text detection, but it is challenging to deal with small targets and solve extremely unbalanced data. We continued to use PSENet to deal with such problems in this work. On the other hand, we studied the problem that most of the existing scene text detection methods use ResNet and FPN as the backbone of feature extraction, and improved the ResNet and FPN network parts of PSENet to make it more conducive to the combination of feature extraction in the early stage. A SEMPANet framework without an anchor and in one stage is proposed to implement a lightweight model, which is embodied in the training time of about 24 h. Finally, we selected the two most representative datasets for oriented text and curved text to conduct experiments. On ICDAR2015, the improved network's latest results further verify its effectiveness; it reached 1.01% in F-measure compared with PSENet-1s. On CTW1500, the improved network performed better than the original network on average.

Darks and Lights, the 'Yin-Yang' of Vision Depends on Luminance

We all know the disappointment when, after a wonderful snapshot, the details in the photo are at much lower contrast than seen before with our own eyes. A recent study by Rahimi-Nasrabadi et al. revealed that this is because human vision accounts for actual luminance range and for accompanied asymmetric changes in dark and light contrasts.

Image Valence Modulates the Processing of Low-Resolution Affective Natural Scenes

In natural vision, noisy and distorted visual inputs often change our perceptual strategy in scene perception. However, it is unclear the extent to which the affective meaning embedded in the degraded natural scenes modulates our scene understanding and associated eye movements. In this eye-tracking experiment by presenting natural scene images with different categories and levels of emotional valence (high-positive, medium-positive, neutral/low-positive, medium-negative, and high-negative), we systematically investigated human participants' perceptual sensitivity (image valence categorization and arousal rating) and image-viewing gaze behaviour to the changes of image resolution. Our analysis revealed that reducing image resolution led to decreased valence recognition and arousal rating, decreased number of fixations in image-viewing but increased individual fixation duration, and stronger central fixation bias. Furthermore, these distortion effects were modulated by the scene valence with less deterioration impact on the valence categorization of negatively valenced scenes and on the gaze behaviour in viewing of high emotionally charged (high-positive and high-negative) scenes. It seems that our visual system shows a valence-modulated susceptibility to the image distortions in scene perception.

Natural-scene-based Steady-state Visual Evoked Potentials Reveal Effects of Short-term Monocular Deprivation

Ocular dominance plasticity beyond the critical period has been demonstrated in adult humans in recent investigations of short-term monocular deprivation (MD). To our knowledge, all previous research adopted non-natural synthetic stimuli in testing perceptual ocular dominance before and after the MD. However, it is recognized that complex natural stimuli may engage cortical mechanisms substantially different from simple synthetic stimuli. Therefore, it remains largely unknown whether reshaping of ocular dominance following MD could be observed during perception of natural scene stimuli without conspicuous interocular competition. Here we used the steady-state visual evoked potential (SSVEP) technique to measure the ocular-specific neural effects of MD with natural scene stimuli where the two eyes' images were tagged with different frequencies. Two hours of MD boosted the neural gain for the deprived eye. During the course of MD, the SSVEP amplitude ratio for the deprived eye compared to the non-deprived eye increased significantly over time, indicating a progressive increase of neural gain for the deprived eye. These findings demonstrate that the effects of short-term MD can manifest when viewing natural scenes, providing a natural case in support of the homeostatic compensation theory of MD. Our work also indicates that the technique of natural-scene-based SSVEP could be particularly useful for future work exploring the neural dynamics during adaptation to natural stimuli.

Automatic encoding of a target position relative to a natural scene

We previously showed that the brain automatically represents a target position for reaching relative to a large square in the background. In the present study, we tested whether a natural scene with many complex details serves as an effective background for representing a target. In the first experiment, we used upright and inverted pictures of a natural scene. A shift of pictures significantly attenuated prism adaptation of reaching movements as long as they were upright. In one-third of participants, adaptation was almost completely cancelled whether the pictures were upright or inverted. It was remarkable that there were two distinct groups of participants, one who relies fully on the allocentric coordinate and the other who depended only when the scene was upright. In the second experiment, we examined how long it takes for a novel upright scene to serve as a background. A shift of the novel scene had no significant effects when it was presented for 500 ms before presenting a target, but significant effects were recovered when presented for 1,500 ms. These results show that a natural scene serves as a background against which a target is automatically represented once we spend 1,500 ms in the scene.NEW & NOTEWORTHY Prism adaptation of reaching was attenuated by a shift of natural scenes as long as they were upright. In one-third of participants, adaptation was fully canceled whether the scene was upright or inverted. When an upright scene was novel, it took 1,500 ms to prepare the scene for allocentric coding. These results show that a natural scene serves as a background against which a target is automatically represented once we spend 1,500 ms in the scene.

Motion contrast in primary visual cortex: a direct comparison of single neuron and population encoding

Features from outside the classical receptive field (CRF) can modulate the stimulus-driven activity of single cells in the primary visual cortex. This modulation, mediated by horizontal and feedback networks, has been extensively described as a variation of firing rate and is considered the basis of processing features as, for example, motion contrast. However, surround influences have also been identified in pairwise spiking or local field coherence. Yet, evidence about co-existence and integration of different neural signatures remains elusive. To compare multiple signatures, we recorded spiking and LFP activity evoked by stimuli exhibiting a motion contrast in the CRFs surround in anesthetized cat primary visual cortex. We chose natural-like scenes over gratings to avoid predominance of simple visual features, which could be easily represented by a rate code. We analyzed firing rates and phase-locking to low-gamma frequency in single cells and neuronal assemblies. Motion contrast was reflected in all measures but in semi-independent populations. Whereas activation of assemblies accompanied single neuron rates, their phase relations were modulated differently. Interestingly, only assembly phase relations mirrored the direction of movement of the surround and were selectively affected by thermal deactivation of visual interhemispheric connections. We argue that motion contrast can be reflected in complementary and superimposed neuronal signatures that can represent different surround features in independent neuronal populations.

Non-uniform phase sensitivity in spatial frequency maps of the human visual cortex

KEY POINTS

Just as a portrait painting can come from a collection of coarse and fine details, natural vision can be decomposed into coarse and fine components. Previous studies have shown that the early visual areas in the brain represent these components in a map-like fashion. Other studies have shown that these same visual areas can be sensitive to how coarse and fine features line up in space. We found that the brain actually jointly represents both the scale of the feature (fine, medium, or coarse) and the alignment of these features in space. The results suggest that the visual cortex has an optimized representation particularly for the alignment of fine details, which are crucial in understanding the visual scene.

ABSTRACT

Complex natural scenes can be decomposed into their oriented spatial frequency (SF) and phase relationships, both of which are represented locally at the earliest stages of cortical visual processing. The SF preference map in the human cortex, obtained using synthetic stimuli, is orderly and correlates strongly with eccentricity. In addition, early visual areas show sensitivity to the phase information that describes the relationship between SFs and thereby dictates the structure of the image. Taken together, two possibilities arise for the joint representation of SF and phase: either the entirety of the cortical SF map is uniformly sensitive to phase, or a particular set of SFs is selectively phase sensitive - for example, greater phase sensitivity for higher SFs that define fine-scale edges in a complex scene. To test between these two possibilities, we constructed a novel continuous natural scene video whereby phase information was maintained in one SF band but scrambled elsewhere. By shifting the central frequency of the phase-aligned band in time, we mapped the phase-sensitive SF preference of the visual cortex. Using functional magnetic resonance imaging, we found that phase sensitivity in early visual areas is biased toward higher SFs. Compared to a SF map of the same scene obtained using linear-filtered stimuli, a much larger patch of areas V1 and V2 is sensitive to the phase alignment of higher SFs. The results of early areas cannot be explained by attention. Our results suggest non-uniform sensitivity to phase alignment in population-level SF representations, with phase alignment being particularly important for fine-scale edge representations of natural scenes.

Integration of egocentric and allocentric information during memory-guided reaching to images of a natural environment

When interacting with our environment we generally make use of egocentric and allocentric object information by coding object positions relative to the observer or relative to the environment, respectively. Bayesian theories suggest that the brain integrates both sources of information optimally for perception and action. However, experimental evidence for egocentric and allocentric integration is sparse and has only been studied using abstract stimuli lacking ecological relevance. Here, we investigated the use of egocentric and allocentric information during memory-guided reaching to images of naturalistic scenes. Participants encoded a breakfast scene containing six objects on a table (local objects) and three objects in the environment (global objects). After a 2 s delay, a visual test scene reappeared for 1 s in which 1 local object was missing (= target) and of the remaining, 1, 3 or 5 local objects or one of the global objects were shifted to the left or to the right. The offset of the test scene prompted participants to reach to the target as precisely as possible. Only local objects served as potential reach targets and thus were task-relevant. When shifting objects we predicted accurate reaching if participants only used egocentric coding of object position and systematic shifts of reach endpoints if allocentric information were used for movement planning. We found that reaching movements were largely affected by allocentric shifts showing an increase in endpoint errors in the direction of object shifts with the number of local objects shifted. No effect occurred when one local or one global object was shifted. Our findings suggest that allocentric cues are indeed used by the brain for memory-guided reaching towards targets in naturalistic visual scenes. Moreover, the integration of egocentric and allocentric object information seems to depend on the extent of changes in the scene.

The neural bases of spatial frequency processing during scene perception

Theories on visual perception agree that scenes are processed in terms of spatial frequencies. Low spatial frequencies (LSF) carry coarse information whereas high spatial frequencies (HSF) carry fine details of the scene. However, how and where spatial frequencies are processed within the brain remain unresolved questions. The present review addresses these issues and aims to identify the cerebral regions differentially involved in low and high spatial frequency processing, and to clarify their attributes during scene perception. Results from a number of behavioral and neuroimaging studies suggest that spatial frequency processing is lateralized in both hemispheres, with the right and left hemispheres predominantly involved in the categorization of LSF and HSF scenes, respectively. There is also evidence that spatial frequency processing is retinotopically mapped in the visual cortex. HSF scenes (as opposed to LSF) activate occipital areas in relation to foveal representations, while categorization of LSF scenes (as opposed to HSF) activates occipital areas in relation to more peripheral representations. Concomitantly, a number of studies have demonstrated that LSF information may reach high-order areas rapidly, allowing an initial coarse parsing of the visual scene, which could then be sent back through feedback into the occipito-temporal cortex to guide finer HSF-based analysis. Finally, the review addresses spatial frequency processing within scene-selective regions areas of the occipito-temporal cortex.

Horizontal information drives the behavioral signatures of face processing

Recent psychophysical evidence indicates that the vertical arrangement of horizontal information is particularly important for encoding facial identity. In this paper we extend this notion to examine the role that information at different (particularly cardinal) orientations might play in a number of established phenomena each a behavioral “signature” of face processing. In particular we consider (a) the face inversion effect (FIE), (b) the facial identity after-effect, (c) face-matching across viewpoint, and (d) interactive, so-called holistic, processing of face parts. We report that filtering faces to remove all but the horizontal information largely preserves these effects but conversely, retaining vertical information generally diminishes or abolishes them. We conclude that preferential processing of horizontal information is a central feature of human face processing that supports many of the behavioral signatures of this critical visual operation.