Assessing Raters: What Factors Predict Discernment in Novice Creativity Raters?

No strong support for a Dunning-Kruger effect in creativity: analyses of self-assessment in absolute and relative terms

Competencies related to the evaluation of own cognitive processes, called metacognitive monitoring, are crucial as they help decide whether to persist in or desist from cognitive efforts. One of the most well-known phenomena in this context-the Dunning-Kruger effect-is that less skilled people tend to overestimate their performance. This effect has been reported for various kinds of performance including creativity. More recently, however, it has been suggested that this phenomenon could be a statistical artifact caused by the better-than-average effect and by regression toward the mean. Therefore, we examined the Dunning-Kruger effect in the context of creative thinking performance (i.e., divergent thinking ability) across two studies (Study 1: N = 425; Study 2: N = 317) and applied the classical quartile-based analysis as well as newly recommended, advanced statistical approaches: the Glejser test of heteroscedasticity and nonlinear quadratic regression. We found that the results indeed depended on the employed statistical method: While classical analyses supported the Dunning-Kruger effect across all conditions, it was not consistently supported by the more advanced statistical methods. These findings are in line with recent work challenging certain assumptions of the Dunning-Kruger effect and we discuss factors that undermine accurate self-assessments, especially in the context of creative performance.

AuDrA: An automated drawing assessment platform for evaluating creativity

The visual modality is central to both reception and expression of human creativity. Creativity assessment paradigms, such as structured drawing tasks Barbot (2018), seek to characterize this key modality of creative ideation. However, visual creativity assessment paradigms often rely on cohorts of expert or naïve raters to gauge the level of creativity of the outputs. This comes at the cost of substantial human investment in both time and labor. To address these issues, recent work has leveraged the power of machine learning techniques to automatically extract creativity scores in the verbal domain (e.g., SemDis; Beaty & Johnson 2021). Yet, a comparably well-vetted solution for the assessment of visual creativity is missing. Here, we introduce AuDrA - an Automated Drawing Assessment platform to extract visual creativity scores from simple drawing productions. Using a collection of line drawings and human creativity ratings, we trained AuDrA and tested its generalizability to untrained drawing sets, raters, and tasks. Across four datasets, nearly 60 raters, and over 13,000 drawings, we found AuDrA scores to be highly correlated with human creativity ratings for new drawings on the same drawing task (r = .65 to .81; mean = .76). Importantly, correlations between AuDrA scores and human raters surpassed those between drawings' elaboration (i.e., ink on the page) and human creativity raters, suggesting that AuDrA is sensitive to features of drawings beyond simple degree of complexity. We discuss future directions, limitations, and link the trained AuDrA model and a tutorial ( https://osf.io/kqn9v/ ) to enable researchers to efficiently assess new drawings.

A systematic framework of creative metacognition

Creative cognition does not just involve cognitive processes in direct service of the main task objective (e.g., idea generation), but also metacognitive processes that monitor and regulate cognition adaptively (e.g., evaluation of ideas and task performance, or development and selection of task strategies). Although metacognition is vital for creative performance, relevant work is sparse, which may be partly due to persistent ambiguities in the theoretical conceptualization of creative metacognition. Therefore, this article proposes a systematic framework of creative metacognition (CMC), which builds on recent advancements in metacognition theory and extends them to meet the specifics of creative cognition. The CMC framework consists of two dynamic components-monitoring and control-and a more static component of metacognitive knowledge, each subsuming metacognitive processes applying to the level of task, performance, and responses. We describe the presumed function of these metacognitive components in the creative process, present evidence in support of each, and discuss their association with related constructs, such as creative self-beliefs. We further highlight the dynamic interplay of metacognitive processes across task performance and identify promising avenues for future research.

Through Thick and Thin: Changes in Creativity During the First Lockdown of the COVID-19 Pandemic

COVID-19 took us by surprise. We all had to face the lockdown and pandemic that put us in a new context, changing our way of life, work conditions, and habits. Coping with such an unprecedented situation may have stimulated creativity. However, the situation also restricted our liberties and triggered health or psychological difficulties. We carried out an online survey (n = 380) to examine whether and how the COVID-19 related first lockdown period was associated with creativity changes in French speaking population. Despite a global negative subjective experience of the situation, participants reported that they were more creative during the lockdown than before. Positive changes were linked with more time availability, more motivation, or the need to solve a problem while negative changes were related to negative affective feelings or a lack of resources or opportunities. This study documents the effects of the first lockdown period on creativity and the factors that influenced it.

Acute and Chronic Physical Activity Increases Creative Ideation Performance: A Systematic Review and Multilevel Meta-analysis

BACKGROUND

Physical activity is a health-relevant lifestyle factor associated with various benefits on physical and mental health. Several meta-analyses indicated effects of acute and chronic physical activities on elementary cognitive functions such as executive control processes, memory, and attention. Meta-analytic evidence on the effects of physical activity on creative idea generation, which involves a conglomerate of these elementary cognitive functions, is largely missing.

OBJECTIVE

A twofold approach was used to evaluate (1) if there is an association between habitual physical activity and creative ideation and (2) if physical activity interventions (acute and chronic) enhance creative ideation performance.

METHODS

Multilevel meta-analytic methods were applied to (1) evaluate the cross-sectional association between creative ideation performance and measures of habitual physical activity and (2) the effect of physical activity on creative ideation performance. Indicators of creative ideation (fluency, flexibility, originality, elaboration, or composite score), creativity domain (verbal, figural), population (adults, children), gender, study quality, and publication year served as moderator variables for both meta-analyses. Analyses of intervention studies additionally examined the moderator variables study design (between, within), time of measurement (during, after), and implementation of intervention (acute, chronic).

RESULTS

The applied meta-analytic multilevel analysis indicated a medium effect for cross-sectional studies (r = 0.22, SE = 0.06, p = 0.002, 95% CI [0.10-0.34]) based on 17 effects sizes from seven studies. The pooled effects of 28 intervention studies, providing 115 effect sizes, indicated a medium effect size of Hedges' g = 0.47 (SE = 0.09, p < 0.001, 95% CI [0.30-0.65]). Furthermore, a stronger effect was observed for chronic interventions of several days or weeks in comparison with acute interventions of one single bout.

CONCLUSION

This study adds important new meta-analytic evidence on the beneficial role of physical activity beyond mental and physical health outcomes: Physical activity has a positive impact on creative ideation, which expands the literature on the role of physical activity in more elementary cognitive functions such as executive control, memory, and attention. Moderator analyses suggested that chronic interventions showed stronger effects than single bouts of physical activity. Rigorously conducted randomized controlled intervention studies and more cross-sectional studies are needed to broaden the evidence in this nascent field of research.