Complexity of Images: Experimental and Computational Estimates Compared

Intracranial study in humans: Neural spectral changes during watching comedy movie of Charlie Chaplin

Humor plays a prominent role in our lives. Thus, understanding the cognitive and neural mechanisms of humor is particularly important. Previous studies that investigated neural substrates of humor used functional MRI and to a lesser extent EEG. In the present study, we conducted intracranial recording in human patients, enabling us to obtain the signal with high temporal precision from within specific brain locations. Our analysis focused on the temporal lobe and the surrounding areas, the temporal lobe was most densely covered in our recording. Thirteen patients watched a fragment of a Charlie Chaplin movie. An independent group of healthy participants rated the same movie fragment, helping us to identify the most funny and the least funny frames of the movie. We compared neural activity occurring during the most funny and least funny frames across frequencies in the range of 1-170 Hz. The most funny compared to least funny parts of the movie were associated with activity modulation in the broadband high-gamma (70-170 Hz; mostly activation) and to a lesser extent gamma band (40-69Hz; activation) and low frequencies (1-12 Hz, delta, theta, alpha bands; mostly deactivation). With regard to regional specificity, we found three types of brain areas: (I) temporal pole, middle and inferior temporal gyrus (both anterior and posterior) in which there was both activation in the high-gamma/gamma bands and deactivation in low frequencies; (II) ventral part of the temporal lobe such as the fusiform gyrus, in which there was mostly deactivation the low frequencies; (III) posterior temporal cortex and its environment, such as the middle occipital and the temporo-parietal junction, in which there was activation in the high-gamma/gamma band. Overall, our results suggest that humor appreciation might be achieved by neural activity across the frequency spectrum.

Familiarity acts as a reduction in objective complexity

High-complexity stimuli are thought to place extra demands on working memory when processing and manipulating such stimuli; however, operational definitions of complexity are not well established, nor are the measures that would demonstrate such effects. Here, we argue that complexity is a relative quantity that is affected by preexisting experience. Experiment 1 compared cued-recall performance for Chinese and English speakers when the stimuli involved Chinese features that varied in the number of strokes or involved Ethiopic features unfamiliar to both groups. Chinese pseudocharacters (two radicals) had half the strokes of Chinese pseudowords (two characters). The response terms were English words familiar to both groups. English speakers performed equivalently with the Ethiopic and pseudocharacters, but much worse on the pseudowords. In contrast, Chinese speakers performed equivalently with pseudowords or pseudocharacters, but worse with Ethiopic cues. Experiment 2 showed that the lack of a complexity effect for Chinese speakers was not due to greater ease of rehearsal of pseudowords compared with pseudocharacters. Experiment 3 ruled out that Chinese speakers are just better at learning paired associates involving Mandarin by demonstrating that while complexity did not affect them, other features of the stimuli did. Taken together, it appears that complexity is not an absolute property based on the number of visual elements, but rather a relative property affected by one's prior knowledge.

Complexity can facilitate visual and auditory perception

Visual and auditory inputs vary in complexity. For example, driving in a city versus the country or listening to the radio versus not are experiences that differ in complexity. How does such complexity impact perception? One possibility is that complex stimuli demand resources that exceed attentional or working memory capacities, reducing sensitivity to perceptual changes. Alternatively, complexity may allow for richer and more distinctive representations, increasing such sensitivity. We performed five experiments to test the nature of the relationship between complexity and perceptual sensitivity during movie clip viewing. Experiment 1 revealed higher sensitivity to global changes in audio or video streams for clips with greater complexity, defined both subjectively (judgments by independent coders) and objectively (information-theoretic redundancy). Experiment 2 replicated this finding but found no evidence that it resulted from complexity drawing attention. Experiment 3 provided a boundary condition by showing that change detection was unaffected by complexity when the changes were superimposed on, rather than dispersed throughout, the clips. Experiment 4 suggested that the effect of complexity, at least when defined objectively, was present without the working memory demands of the preceding experiments. Experiment 5 suggested that complexity led to richer representations of the clips, as reflected in enhanced long-term memory. Collectively, these findings show that, despite increasing informational load, complexity can serve to ground and facilitate perceptual sensitivity. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Bandwidth Modeling of Silicon Retinas for Next Generation Visual Sensor Networks

Silicon retinas, also known as Dynamic Vision Sensors (DVS) or event-based visual sensors, have shown great advantages in terms of low power consumption, low bandwidth, wide dynamic range and very high temporal resolution. Owing to such advantages as compared to conventional vision sensors, DVS devices are gaining more and more attention in various applications such as drone surveillance, robotics, high-speed motion photography, etc. The output of such sensors is a sequence of events rather than a series of frames as for classical cameras. Estimating the data rate of the stream of events associated with such sensors is needed for the appropriate design of transmission systems involving such sensors. In this work, we propose to consider information about the scene content and sensor speed to support such estimation, and we identify suitable metrics to quantify the complexity of the scene for this purpose. According to the results of this study, the event rate shows an exponential relationship with the metric associated with the complexity of the scene and linear relationships with the speed of the sensor. Based on these results, we propose a two-parameter model for the dependency of the event rate on scene complexity and sensor speed. The model achieves a prediction accuracy of approximately 88.4% for the outdoor environment along with the overall prediction performance of approximately 84%.

Eye-movement behaviours when viewing real-world pain-related images

BACKGROUND

Pain-related cues are evolutionarily primed to capture attention, although evidence of attentional biases towards pain-related information is mixed in healthy individuals. The present study explores whether healthy individuals show significantly different eye-movement behaviours when viewing real-world pain-related scenes compared to neutral scenes. The effect of manipulating via written information the threat value of the pain-related scenes on eye-movement behaviours was also assessed.

METHODS

Participants were randomized to threatening (n = 28) and non-threatening (n = 27) information conditions. All completed a free-viewing task with real-world pain-related and neutral images while their eye movements were recorded.

RESULTS

Participants made significantly fewer fixations of significantly longer duration when viewing pain-related images compared to neutral images. No significant differences were found between threatening and non-threatening information groups in their pattern of eye movements.

CONCLUSIONS

This study shows that healthy individuals demonstrate attentional biases to pain-related real-world complex images compared to neutral images. Future research is needed to establish the implications of these biases, particularly in the context of acute pain, on the onset and/or subsequent maintenance of chronic pain conditions.

SIGNIFICANCE

Healthy individuals show different eye-movement behaviours when viewing pain-related scenes than neutral scenes, supporting evolutionary accounts of pain. Implications for the onset and/or maintenance of chronic pain need to be explored.

Head Motion Elicited by Viewing Affective Pictures as Measured by a New LED-Based Technique

The complex sensory input and motor reflexes that keep body posture and head position aligned are influenced by emotional reactions evoked by visual or auditory stimulation. Several theoretical approaches have emphasized the relevance of motor reactions in emotional response. Emotions are considered as a tendency or predisposition to act that depends on two motivational systems in the brain - the appetitive system, related to approach behaviours, and the defensive system, related to withdrawal or fight-or-flight behaviours. Few studies on emotion have been conducted employing kinematic methods, however. Motion analysis of the head may be a promising method for studying the impact of viewing affective pictures on emotional response. For this purpose, we presented unpleasant, neutral and pleasant affective pictures. Participants were instructed to view the pictures and to remain still. Two light-emitting diodes (LEDs) were attached to the foreheads of participants, and a Wii Remote controller, positioned 25 cm away, detected the position of the LEDs in the medial-lateral and anterior-posterior axes. We found more sway in response to unpleasant pictures. In addition, unpleasant pictures also provoked faster movements than both neutral and pleasant pictures. This response to unpleasant pictures, in contrast to pleasant ones, might reflect the readiness or predisposition to act. Our data also revealed that men moved faster than women, which is in accordance with previous findings related to gender differences.

GraphCom: A multidimensional measure of graphic complexity applied to 131 written languages

We report a new multidimensional measure of visual complexity (GraphCom) that captures variability in the complexity of graphs within and across writing systems. We applied the measure to 131 written languages, allowing comparisons of complexity and providing a basis for empirical testing of GraphCom. The measure includes four dimensions whose value in capturing the different visual properties of graphs had been demonstrated in prior reading research-(1) perimetric complexity, sensitive to the ratio of a written form to its surrounding white space (Pelli, Burns, Farell, & Moore-Page, 2006); (2) number of disconnected components, sensitive to discontinuity (Gibson, 1969); (3) number of connected points, sensitive to continuity (Lanthier, Risko, Stolz, & Besner, 2009); and (4) number of simple features, sensitive to the strokes that compose graphs (Wu, Zhou, & Shu, 1999). In our analysis of the complexity of 21,550 graphs, we (a) determined the complexity variation across writing systems along each dimension, (b) examined the relationships among complexity patterns within and across writing systems, and (c) compared the dimensions in their abilities to differentiate the graphs from different writing systems, in order to predict human perceptual judgments (n = 180) of graphs with varying complexity. The results from the computational and experimental comparisons showed that GraphCom provides a measure of graphic complexity that exceeds previous measures in its empirical validation. The measure can be universally applied across writing systems, providing a research tool for studies of reading and writing.

Predicting perceived visual complexity of abstract patterns using computational measures: The influence of mirror symmetry on complexity perception

Visual complexity is relevant for many areas ranging from improving usability of technical displays or websites up to understanding aesthetic experiences. Therefore, many attempts have been made to relate objective properties of images to perceived complexity in artworks and other images. It has been argued that visual complexity is a multidimensional construct mainly consisting of two dimensions: A quantitative dimension that increases complexity through number of elements, and a structural dimension representing order negatively related to complexity. The objective of this work is to study human perception of visual complexity utilizing two large independent sets of abstract patterns. A wide range of computational measures of complexity was calculated, further combined using linear models as well as machine learning (random forests), and compared with data from human evaluations. Our results confirm the adequacy of existing two-factor models of perceived visual complexity consisting of a quantitative and a structural factor (in our case mirror symmetry) for both of our stimulus sets. In addition, a non-linear transformation of mirror symmetry giving more influence to small deviations from symmetry greatly increased explained variance. Thus, we again demonstrate the multidimensional nature of human complexity perception and present comprehensive quantitative models of the visual complexity of abstract patterns, which might be useful for future experiments and applications.

Grounding by Attention Simulation in Peripersonal Space: Pupils Dilate to Pinch Grip But Not Big Size Nominal Classifier

Grammatical categories represent implicit knowledge, and it is not known if such abstract linguistic knowledge can be continuously grounded in real-life experiences, nor is it known what types of mental states can be simulated. A former study showed that attention bias in peripersonal space (PPS) affects reaction times in grammatical congruency judgments of nominal classifiers, suggesting that simulated semantics may include reenactment of attention. In this study, we contrasted a Chinese nominal classifier used with nouns denoting pinch grip objects with a classifier for nouns with big object referents in a pupil dilation experiment. Twenty Chinese native speakers read grammatical and ungrammatical classifier-noun combinations and made grammaticality judgment while their pupillary responses were measured. It was found that their pupils dilated significantly more to the pinch grip classifier than to the big object classifier, indicating attention simulation in PPS. Pupil dilations were also significantly larger with congruent trials on the whole than in incongruent trials, but crucially, congruency and classifier semantics were independent of each other. No such effects were found in controls.

Effects of proactive interference on non-verbal working memory

Working memory (WM) is a cognitive system responsible for actively maintaining and processing relevant information and is central to successful cognition. A process critical to WM is the resolution of proactive interference (PI), which involves suppressing memory intrusions from prior memories that are no longer relevant. Most studies that have examined resistance to PI in a process-pure fashion used verbal material. By contrast, studies using non-verbal material are scarce, and it remains unclear whether the effect of PI is domain-general or whether it applies solely to the verbal domain. The aim of the present study was to examine the effect of PI in visual WM using both objects with high and low nameability. Using a Directed-Forgetting paradigm, we varied discriminability between WM items on two dimensions, one verbal (high-nameability vs. low-nameability objects) and one perceptual (colored vs. gray objects). As in previous studies using verbal material, effects of PI were found with object stimuli, even after controlling for verbal labels being used (i.e., low-nameability condition). We also found that the addition of distinctive features (color, verbal label) increased performance in rejecting intrusion probes, most likely through an increase in discriminability between content-context bindings in WM.

Predicting Complexity Perception of Real World Images

The aim of this work is to predict the complexity perception of real world images. We propose a new complexity measure where different image features, based on spatial, frequency and color properties are linearly combined. In order to find the optimal set of weighting coefficients we have applied a Particle Swarm Optimization. The optimal linear combination is the one that best fits the subjective data obtained in an experiment where observers evaluate the complexity of real world scenes on a web-based interface. To test the proposed complexity measure we have performed a second experiment on a different database of real world scenes, where the linear combination previously obtained is correlated with the new subjective data. Our complexity measure outperforms not only each single visual feature but also two visual clutter measures frequently used in the literature to predict image complexity. To analyze the usefulness of our proposal, we have also considered two different sets of stimuli composed of real texture images. Tuning the parameters of our measure for this kind of stimuli, we have obtained a linear combination that still outperforms the single measures. In conclusion our measure, properly tuned, can predict complexity perception of different kind of images.

Effects of presentation duration on measures of complexity in affective environmental scenes and representational paintings

Complexity constitutes an integral part of humans' environment and is inherent to information processing. However, little is known about the dynamics of visual complexity perception of affective environmental scenes (IAPS pictures) and artworks, such as affective representational paintings. In three experiments, we studied the time course of visual complexity perception by varying presentation duration and comparing subjective ratings with objective measures of complexity. In Experiment 1, 60 females rated 96 IAPS pictures, presented either for 1, 5, or 25s, for familiarity, complexity, pleasantness and arousal. In Experiment 2, another 60 females rated 96 representational paintings. Mean ratings of complexity and pleasantness changed according to presentation duration in a similar vein in both experiments, suggesting an inverted U-shape. No common pattern of results was observed for arousal and familiarity ratings across the two picture sets. The correlations between subjective and objective measures of complexity increased with longer exposure durations for IAPS pictures, but results were more ambiguous for paintings. Experiment 3 explored the time course of the multidimensionality of visual complexity perception. Another 109 females rated the number of objects, their disorganization and the differentiation between a figure-ground vs. complex scene composition of pictures presented for 1 and 5s. The multidimensionality of visual complexity only clearly emerged in the 5-s condition. In both picture sets, the strength of the correlations with objective measures depended on the type of subdimension of complexity and was less affected by presentation duration than correlations with general complexity in Experiments 1 and 2. These results have clear implications for perceptual and cognitive theories, especially for those of esthetic experiences, in which the dynamical changes of complexity perception need to be integrated.

Visual Complexity Analysis Using Taxonomic Diagrams of Figures and Backgrounds in Japanese Residential Streetscapes

This study was conducted with the objective of finding the perceivable effect of figures and backgrounds on residential streetscapes and their connections in visual complexity. The visual complexity depends on the extent of information a viewer can observe from a visible area. The information includes a number of visual elements along the streetscapes and their diversity and interconnections. The aim of this research was to analyze the structural hierarchical visual complexity of the streetscapes, caused by varied spatial arrangement and numerous spatial connections of the perceivable visual elements. The visual elements of 60 residential streetscapes were classified into figures and backgrounds using human perception of 20 subjects. The identified figures and backgrounds were arranged in a taxonomic diagram representing their connections. These taxonomic diagrams reflect the structural hierarchical visual complexity. Finally, taxonomic entropy was applied to statistically analyze the structural hierarchical visual complexity. When the taxonomic diagram is vertically and horizontally lengthy and the arrangement of the elements of taxonomic diagram is irregular, the complexity increases depicting a large number of figures whose spatial connections impart a high visual complexity to the streetscapes.

A Relationship Between Visual Complexity and Aesthetic Appraisal of Car Front Images: An Eye-Tracker Study

Image aesthetic pleasure (AP) is conjectured to be related to image visual complexity (VC). The aim of the present study was to investigate whether (a) two image attributes, AP and VC, are reflected in eye-movement parameters; and (b) subjective measures of AP and VC are related. Participants (N=26) explored car front images (M=50) while their eye movements were recorded. Following image exposure (10 seconds), its VC and AP were rated. Fixation count was found to positively correlate with the subjective VC and its objective proxy, JPEG compression size, suggesting that this eye-movement parameter can be considered an objective behavioral measure of VC. AP, in comparison, positively correlated with average dwelling time. Subjective measures of AP and VC were related too, following an inverted U-shape function best-fit by a quadratic equation. In addition, AP was found to be modulated by car prestige. Our findings reveal a close relationship between subjective and objective measures of complexity and aesthetic appraisal, which is interpreted within a prototype-based theory framework.

The Role of Shape in Semantic Memory Organization of Objects: An Experimental Study Using PI-Release

Visual information contributes fundamentally to the process of object categorization. The present study investigated whether the degree of activation of visual information in this process is dependent on the contextual relevance of this information. We used the Proactive Interference (PI-release) paradigm. In four experiments, we manipulated the information by which objects could be categorized and subsequently be retrieved from memory. The pattern of PI-release showed that if objects could be stored and retrieved both by (non-perceptual) semantic and (perceptual) shape information, then shape information was overruled by semantic information. If, however, semantic information could not be (satisfactorily) used to store and retrieve objects, then objects were stored in memory in terms of their shape. The latter effect was found to be strongest for objects from identical semantic categories.

Measuring streetscape complexity based on the statistics of local contrast and spatial frequency

Streetscapes are basic urban elements which play a major role in the livability of a city. The visual complexity of streetscapes is known to influence how people behave in such built spaces. However, how and which characteristics of a visual scene influence our perception of complexity have yet to be fully understood. This study proposes a method to evaluate the complexity perceived in streetscapes based on the statistics of local contrast and spatial frequency. Here, 74 streetscape images from four cities, including daytime and nighttime scenes, were ranked for complexity by 40 participants. Image processing was then used to locally segment contrast and spatial frequency in the streetscapes. The statistics of these characteristics were extracted and later combined to form a single objective measure. The direct use of statistics revealed structural or morphological patterns in streetscapes related to the perception of complexity. Furthermore, in comparison to conventional measures of visual complexity, the proposed objective measure exhibits a higher correlation with the opinion of the participants. Also, the performance of this method is more robust regarding different time scenarios.

Examining complexity across domains: relating subjective and objective measures of affective environmental scenes, paintings and music

Subjective complexity has been found to be related to hedonic measures of preference, pleasantness and beauty, but there is no consensus about the nature of this relationship in the visual and musical domains. Moreover, the affective content of stimuli has been largely neglected so far in the study of complexity but is crucial in many everyday contexts and in aesthetic experiences. We thus propose a cross-domain approach that acknowledges the multidimensional nature of complexity and that uses a wide range of objective complexity measures combined with subjective ratings. In four experiments, we employed pictures of affective environmental scenes, representational paintings, and Romantic solo and chamber music excerpts. Stimuli were pre-selected to vary in emotional content (pleasantness and arousal) and complexity (low versus high number of elements). For each set of stimuli, in a between-subjects design, ratings of familiarity, complexity, pleasantness and arousal were obtained for a presentation time of 25 s from 152 participants. In line with Berlyne's collative-motivation model, statistical analyses controlling for familiarity revealed a positive relationship between subjective complexity and arousal, and the highest correlations were observed for musical stimuli. Evidence for a mediating role of arousal in the complexity-pleasantness relationship was demonstrated in all experiments, but was only significant for females with regard to music. The direction and strength of the linear relationship between complexity and pleasantness depended on the stimulus type and gender. For environmental scenes, the root mean square contrast measures and measures of compressed file size correlated best with subjective complexity, whereas only edge detection based on phase congruency yielded equivalent results for representational paintings. Measures of compressed file size and event density also showed positive correlations with complexity and arousal in music, which is relevant for the discussion on which aspects of complexity are domain-specific and which are domain-general.