A deep-learning framework for human perception of abstract art composition

The perceptual primacy of feeling: Affectless visual machines explain a majority of variance in human visually evoked affect

Looking at the world often involves not just seeing things, but feeling things. Modern feedforward machine vision systems that learn to perceive the world in the absence of active physiology, deliberative thought, or any form of feedback that resembles human affective experience offer tools to demystify the relationship between seeing and feeling, and to assess how much of visually evoked affective experiences may be a straightforward function of representation learning over natural image statistics. In this work, we deploy a diverse sample of 180 state-of-the-art deep neural network models trained only on canonical computer vision tasks to predict human ratings of arousal, valence, and beauty for images from multiple categories (objects, faces, landscapes, art) across two datasets. Importantly, we use the features of these models without additional learning, linearly decoding human affective responses from network activity in much the same way neuroscientists decode information from neural recordings. Aggregate analysis across our survey, demonstrates that predictions from purely perceptual models explain a majority of the explainable variance in average ratings of arousal, valence, and beauty alike. Finer-grained analysis within our survey (e.g. comparisons between shallower and deeper layers, or between randomly initialized, category-supervised, and self-supervised models) point to rich, preconceptual abstraction (learned from diversity of visual experience) as a key driver of these predictions. Taken together, these results provide further computational evidence for an information-processing account of visually evoked affect linked directly to efficient representation learning over natural image statistics, and hint at a computational locus of affective and aesthetic valuation immediately proximate to perception.

How deep is your art: An experimental study on the limits of artistic understanding in a single-task, single-modality neural network

Computational modeling of artwork meaning is complex and difficult. This is because art interpretation is multidimensional and highly subjective. This paper experimentally investigated the degree to which a state-of-the-art Deep Convolutional Neural Network (DCNN), a popular Machine Learning approach, can correctly distinguish modern conceptual art work into the galleries devised by art curators. Two hypotheses were proposed to state that the DCNN model uses Exhibited Properties for classification, like shape and color, but not Non-Exhibited Properties, such as historical context and artist intention. The two hypotheses were experimentally validated using a methodology designed for this purpose. VGG-11 DCNN pre-trained on ImageNet dataset and discriminatively fine-tuned was trained on handcrafted datasets designed from real-world conceptual photography galleries. Experimental results supported the two hypotheses showing that the DCNN model ignores Non-Exhibited Properties and uses only Exhibited Properties for artwork classification. This work points to current DCNN limitations, which should be addressed by future DNN models.

Universality and superiority in preference for chromatic composition of art paintings

Color composition in paintings is a critical factor affecting observers’ aesthetic judgments. We examined observers’ preferences for the color composition of Japanese and Occidental paintings when their color gamut was rotated. In the experiment, observers were asked to select their preferred image from original and three hue-rotated images in a four-alternative forced choice paradigm. Despite observers’ being unfamiliar with the presented artwork, the original paintings (0 degrees) were preferred more frequently than the hue-rotated ones. Furthermore, the original paintings’ superiority was observed when the images were divided into small square pieces and their positions randomized (Scrambled condition), and when the images were composed of square pieces collected from different art paintings and composed as patchwork images (Patchwork condition). Therefore, the original paintings’ superiority regarding preference was quite robust, and the specific objects in the paintings associated with a particular color played only a limited role. Rather, the original paintings’ general trend in color statistics influenced hue-angle preference. Art paintings likely share common statistical regulations in color distributions, which may be the basis for the universality and superiority of the preference for original paintings.