Warm, lively, rough? Assessing agreement on aesthetic effects of artworks

Using machine learning to predict judgments on Western visual art along content-representational and formal-perceptual attributes

Art research has long aimed to unravel the complex associations between specific attributes, such as color, complexity, and emotional expressiveness, and art judgments, including beauty, creativity, and liking. However, the fundamental distinction between attributes as inherent characteristics or features of the artwork and judgments as subjective evaluations remains an exciting topic. This paper reviews the literature of the last half century, to identify key attributes, and employs machine learning, specifically Gradient Boosted Decision Trees (GBDT), to predict 13 art judgments along 17 attributes. Ratings from 78 art novice participants were collected for 54 Western artworks. Our GBDT models successfully predicted 13 judgments significantly. Notably, judged creativity and disturbing/irritating judgments showed the highest predictability, with the models explaining 31% and 32% of the variance, respectively. The attributes emotional expressiveness, valence, symbolism, as well as complexity emerged as consistent and significant contributors to the models' performance. Content-representational attributes played a more prominent role than formal-perceptual attributes. Moreover, we found in some cases non-linear relationships between attributes and judgments with sudden inclines or declines around medium levels of the rating scales. By uncovering these underlying patterns and dynamics in art judgment behavior, our research provides valuable insights to advance the understanding of aesthetic experiences considering visual art, inform cultural practices, and inspire future research in the field of art appreciation.

Cross-cultural comparison of beauty judgments in visual art using machine learning analysis of art attribute predictors among Japanese and German speakers

In empirical art research, understanding how viewers judge visual artworks as beautiful is often explored through the study of attributes-specific inherent characteristics or artwork features such as color, complexity, and emotional expressiveness. These attributes form the basis for subjective evaluations, including the judgment of beauty. Building on this conceptual framework, our study examines the beauty judgments of 54 Western artworks made by native Japanese and German speakers, utilizing an extreme randomized trees model-a data-driven machine learning approach-to investigate cross-cultural differences in evaluation behavior. Our analysis of 17 attributes revealed that visual harmony, color variety, valence, and complexity significantly influenced beauty judgments across both cultural cohorts. Notably, preferences for complexity diverged significantly: while the native Japanese speakers found simpler artworks as more beautiful, the native German speakers evaluated more complex artworks as more beautiful. Further cultural distinctions were observed: for the native German speakers, emotional expressiveness was a significant factor, whereas for the native Japanese speakers, attributes such as brushwork, color world, and saturation were more impactful. Our findings illuminate the nuanced role that cultural context plays in shaping aesthetic judgments and demonstrate the utility of machine learning in unravelling these complex dynamics. This research not only advances our understanding of how beauty is judged in visual art-considering self-evaluated attributes-across different cultures but also underscores the potential of machine learning to enhance our comprehension of the aesthetic evaluation of visual artworks.

"Warm," "cool," and the colors

Participants judged affective cooler/warmer gradients around a 12-step color circle. Each pair of adjacent colors was presented twice (left-right reversed), all in random order. Participants readily performed the task, but their settings do not correlate very well. Individual responses were compared with a small number of canonical templates. For a little less than one-half of the participants responses or judgements correlate with such a template. We find a warm pole (in the orange environment) and a cool pole (in the teal environment) connected with two tracks that tend to have one or more gaps or weak, even inverted links. We conclude that the common artistic cool-warm polarity is only weakly reflected in responses of our observers. If it does, the observers apparently use categorical warm and cool poles and may be uncertain in relating adjacent hue steps along the 12-step color circle.

What kind of impacts can artwork have on viewers? Establishing a taxonomy for aesthetic impacts

What kinds of impacts can visual art have on a viewer? To identify potential art impacts, we recruited five aesthetics experts from different academic disciplines: art history, neuroscience, philosophy, psychology and theology. Together, the group curated a set of terms that corresponded to descriptive features (124 terms) and cognitive-affective impacts (69 terms) of artworks. Using these terms as prompts, participants (n = 899) were given one minute to generate words for each term related to how an artwork looked (descriptive features) or made them think or feel (cognitive-affective impacts). Using network psychometric approaches, we identified terms that were semantically similar based on participants' responses and applied hierarchical exploratory graph analysis to map the relationships between the terms. Our analyses identified 17 descriptive dimensions, which could be further reduced to 5, and 11 impact dimensions, which could be further reduced to 4. The resulting taxonomy demonstrated overlap between the descriptive and impact networks as well as consistency with empirical evidence. This taxonomy could serve as the foundation to empirically evaluate art's impacts on viewers.

Can we really 'read' art to see the changing brain? A review and empirical assessment of clinical case reports and published artworks for systematic evidence of quality and style changes linked to damage or neurodegenerative disease

The past three decades have seen multiple reports of people with neurodegenerative disorders, or other forms of changes in their brains, who also show putative changes in how they approach and produce visual art. Authors argue that these cases may provide a unique body of evidence, so-called 'artistic signatures' of neurodegenerative diseases, that might be used to understand disorders, provide diagnoses, be employed in treatment, create patterns of testable hypotheses for causative study, and also provide unique insight into the neurobiological linkages between the mind, brain, body, and the human penchant for art-making itself. However-before we can begin to meaningfully build from such emerging findings, much less formulate applications-not only is such evidence currently quite disparate and in need of systematic review, almost all case reports and artwork ratings are entirely subjective, based on authors' personal observations or a sparse collection of methods that may not best fit underlying research aims. This leads to the very real question of whether we might actually find patterns of systematic change if fit to a rigorous review-Can we really 'read' art to illuminate possible changes in the brain? How might we best approach this topic in future neuroscientific, clinical, and art-related research? This paper presents a review of this field and answer to these questions. We consider the current case reports for seven main disorders-Alzheimer's and Parkinson's disease, frontotemporal and Lewy body dementia, corticobasal degeneration, aphasia, as well as stroke-consolidating arguments for factors and changes related to art-making and critiquing past methods. Taking the published artworks from these papers, we then conduct our own assessment, employing computerized and human-rater-based approaches, which we argue represent best practice to identify stylistic or creativity/quality changes. We suggest, indeed, some evidence for systematic patterns in art-making for specific disorders and also find that case authors showed rather high agreement with our own assessments. More important, through opening this topic and past evidence to a systematic review, we hope to open a discussion and provide a theoretical and empirical foundation for future application and research on the intersection of art-making and the neurotypical, the changed, and the artistic brain.

A data-driven, hyper-realistic method for visualizing individual mental representations of faces

Research in person and face perception has broadly focused on group-level consensus that individuals hold when making judgments of others (e.g., “X type of face looks trustworthy”). However, a growing body of research demonstrates that individual variation is larger than shared, stimulus-level variation for many social trait judgments. Despite this insight, little research to date has focused on building and explaining individual models of face perception. Studies and methodologies that have examined individual models are limited in what visualizations they can reliably produce to either noisy and blurry or computer avatar representations. Methods that produce low-fidelity visual representations inhibit generalizability by being clearly computer manipulated and produced. In the present work, we introduce a novel paradigm to visualize individual models of face judgments by leveraging state-of-the-art computer vision methods. Our proposed method can produce a set of photorealistic face images that correspond to an individual's mental representation of a specific attribute across a variety of attribute intensities. We provide a proof-of-concept study which examines perceived trustworthiness/untrustworthiness and masculinity/femininity. We close with a discussion of future work to substantiate our proposed method.

The Effects of Pitch Manipulation on Male Ratings of Female Speakers and Their Voices

Vocal and facial cues typically co-occur in natural settings, and multisensory processing of voice and face relies on their synchronous presentation. Psychological research has examined various facial and vocal cues to attractiveness as well as to judgements of sexual dimorphism, health, and age. However, few studies have investigated the interaction of vocal and facial cues in attractiveness judgments under naturalistic conditions using dynamic, ecologically valid stimuli. Here, we used short videos or audio tracks of females speaking full sentences and used a manipulation of voice pitch to investigate cross-modal interactions of voice pitch on facial attractiveness and related ratings. Male participants had to rate attractiveness, femininity, age, and health of synchronized audio-video recordings or voices only, with either original or modified voice pitch. We expected audio stimuli with increased voice pitch to be rated as more attractive, more feminine, healthier, and younger. If auditory judgements cross-modally influence judgements of facial attributes, we additionally expected the voice pitch manipulation to affect ratings of audiovisual stimulus material. We tested 106 male participants in a within-subject design in two sessions. Analyses revealed that voice recordings with increased voice pitch were perceived to be more feminine and younger, but not more attractive or healthier. When coupled with video recordings, increased pitch lowered perceived age of faces, but did not significantly influence perceived attractiveness, femininity, or health. Our results suggest that our manipulation of voice pitch has a measurable impact on judgements of femininity and age, but does not measurably influence vocal and facial attractiveness in naturalistic conditions.

Impact of Visual Design Elements and Principles in Human Electroencephalogram Brain Activity Assessed with Spectral Methods and Convolutional Neural Networks

The visual design elements and principles (VDEPs) can trigger behavioural changes and emotions in the viewer, but their effects on brain activity are not clearly understood. In this paper, we explore the relationships between brain activity and colour (cold/warm), light (dark/bright), movement (fast/slow), and balance (symmetrical/asymmetrical) VDEPs. We used the public DEAP dataset with the electroencephalogram signals of 32 participants recorded while watching music videos. The characteristic VDEPs for each second of the videos were manually tagged for by a team of two visual communication experts. Results show that variations in the light/value, rhythm/movement, and balance in the music video sequences produce a statistically significant effect over the mean absolute power of the Delta, Theta, Alpha, Beta, and Gamma EEG bands (p < 0.05). Furthermore, we trained a Convolutional Neural Network that successfully predicts the VDEP of a video fragment solely by the EEG signal of the viewer with an accuracy ranging from 0.7447 for Colour VDEP to 0.9685 for Movement VDEP. Our work shows evidence that VDEPs affect brain activity in a variety of distinguishable ways and that a deep learning classifier can infer visual VDEP properties of the videos from EEG activity.

A deep-learning framework for human perception of abstract art composition

Artistic composition (the structural organization of pictorial elements) is often characterized by some basic rules and heuristics, but art history does not offer quantitative tools for segmenting individual elements, measuring their interactions and related operations. To discover whether a metric description of this kind is even possible, we exploit a deep-learning algorithm that attempts to capture the perceptual mechanism underlying composition in humans. We rely on a robust behavioral marker with known relevance to higher-level vision: orientation judgements, that is, telling whether a painting is hung “right-side up.” Humans can perform this task, even for abstract paintings. To account for this finding, existing models rely on “meaningful” content or specific image statistics, often in accordance with explicit rules from art theory. Our approach does not commit to any such assumptions/schemes, yet it outperforms previous models and for a larger database, encompassing a wide range of painting styles. Moreover, our model correctly reproduces human performance across several measurements from a new web-based experiment designed to test whole paintings, as well as painting fragments matched to the receptive-field size of different depths in the model. By exploiting this approach, we show that our deep learning model captures relevant characteristics of human orientation perception across styles and granularities. Interestingly, the more abstract the painting, the more our model relies on extended spatial integration of cues, a property supported by deeper layers.

A Quantitative Analysis of the Taxonomy of Artistic Styles

Classifying artists and their work as distinct art styles has been an important task of scholars in the field of art history. Due to its subjectivity, scholars often contradict one another. Our project investigated differences in aesthetic qualities of seven art styles through quantitative means. This was achieved with state-of-the-art deep-learning paradigms to generate new images resembling the style of an artist or entire era. We conducted psychological experiments to measure the behavior of subjects when viewing these new art images. Two different experiments were used: In an eye-tracking study, subjects viewed art-stylespecific generated images. Eye movements were recorded and then compared between art styles. In a visual singleton search study, subjects had to locate a style-outlier image among three images of an alternative style. Reaction time and accuracy were measured and analyzed. These experiments show that there are measurable differences in behavior when viewing images of varying art styles. From these differences, we constructed hierarchical clusterings relating art styles based on the different behaviors of subjects viewing the samples. Our study reveals a novel perspective on the classification of artworks into stylistic eras and motivates future research in the domain of empirical aesthetics through quantitative means.