Production and perception rules underlying visual patterns: effects of symmetry and hierarchy

The Order & Complexity Toolbox for Aesthetics (OCTA): A systematic approach to study the relations between order, complexity, and aesthetic appreciation

Do individuals prefer stimuli that are ordered or disordered, simple or complex, or that strike the right balance of order and complexity? Earlier research mainly focused on the separate influence of order and complexity on aesthetic appreciation. When order and complexity were studied in combination, stimulus manipulations were often not parametrically controlled, only rather specific types of order (i.e., balance or symmetry) were usually studied, and/or the multidimensionality of order and complexity was largely ignored. Progress has also been limited by the lack of an easy way to create reproducible and expandible stimulus sets, including both order and complexity manipulations. The Order & Complexity Toolbox for Aesthetics (OCTA), a Python toolbox that is also available as a point-and-click Shiny application, aims to fill this gap. OCTA provides researchers with a free and easy way to create multi-element displays varying qualitatively (i.e., different types) and quantitatively (i.e., different levels) in order and complexity, based on regularity and variety along multiple element features (e.g., shape, size, color, orientation). The standard vector-based output is ideal for experiments on the web and the creation of dynamic interfaces and stimuli. OCTA will not only facilitate reproducible stimulus construction and experimental design in research on order, complexity, and aesthetics. In addition, OCTA can be a very useful tool in any type of research using visual stimuli, or even to create digital art. To illustrate OCTA's potential, we propose several possible applications and diverse questions that can be addressed using OCTA.

Origins of Hierarchical Logical Reasoning

Hierarchical cognitive mechanisms underlie sophisticated behaviors, including language, music, mathematics, tool-use, and theory of mind. The origins of hierarchical logical reasoning have long been, and continue to be, an important puzzle for cognitive science. Prior approaches to hierarchical logical reasoning have often failed to distinguish between observable hierarchical behavior and unobservable hierarchical cognitive mechanisms. Furthermore, past research has been largely methodologically restricted to passive recognition tasks as compared to active generation tasks that are stronger tests of hierarchical rules. We argue that it is necessary to implement learning studies in humans, non-human species, and machines that are analyzed with formal models comparing the contribution of different cognitive mechanisms implicated in the generation of hierarchical behavior. These studies are critical to advance theories in the domains of recursion, rule-learning, symbolic reasoning, and the potentially uniquely human cognitive origins of hierarchical logical reasoning.

Seven-month-old infants detect symmetrical structures in multi-featured abstract visual patterns

The present study investigated 7-month-old infants' ability to perceive structural symmetry in mosaic-like abstract visual patterns. We examined infants' (n = 98) spontaneous looking behaviour to mosaic-like sequences with symmetrical and asymmetrical structures. Sequences were composed of square tiles from two categories that differed in their colour scheme and internal shape. We manipulated sequence length (3 or 5 tiles) and abstractness of the symmetry (token vs. category level). The 7-month-olds discriminated structurally symmetrical from asymmetrical mosaics in the first half of the test phase (first 8 trials). Sequence length, level of symmetry, or number of unique tiles per sequence did not significantly modulate infants' looking behaviour. These results suggest that very young infants detect differences in structural symmetry in multi-featured visual patterns.

Perceptual grouping and detection of trial-unique emergent structures by pigeons

Detecting global patterns in the environment is essential to object perception and recognition. Consistent with this, pigeons have been shown to readily detect and locate geometrically arranged, structured targets embedded in randomized backgrounds. Here we show for the first time that pigeons can detect and localize trial-unique targets derived solely from global patterns resulting from periodicity, symmetry and their combination using randomly generated segments of black and white local elements. The results indicate pigeons can perceptually segment and detect a wide variety of emergent global structures and do so even when they are unique to each trial. The perceptual and cognitive mechanisms underlying this discrimination likely play important roles in the abilities of how pigeons, and likely other birds, detect and categorize the properties of natural objects at different spatial scales.

The Perceived Beauty of Convex Polygon Tilings: The Influence of Regularity, Curvature, and Density

Polygon tilings in natural and man-made objects show great variety. Unlike previous studies that have mainly focused on their classification and production methods, this study aimed at exploring factors that may contribute to the perceived beauty of convex polygon tilings. We analyze the dimensions of regularity, curvature, and density, as well as individual differences. Triangle tilings and hexagon tilings were tested in Experiment 1 and 2, respectively. The results showed that the perceived beauty of convex polygon tilings can be enhanced by higher levels of regularity and nonobvious local curvature. Surprisingly, the effect of density appeared to be different, with the dense triangle tilings and the less dense hexagon tilings scoring higher than the reverse. We discuss a possible explanation based on trypophobia caused by different types of polygons, as well as the observers' personality trait of agreeableness.

Sensitivity to geometric shape regularity in humans and baboons: A putative signature of human singularity

Among primates, humans are special in their ability to create and manipulate highly elaborate structures of language, mathematics, and music. Here we show that this sensitivity to abstract structure is already present in a much simpler domain: the visual perception of regular geometric shapes such as squares, rectangles, and parallelograms. We asked human subjects to detect an intruder shape among six quadrilaterals. Although the intruder was always defined by an identical amount of displacement of a single vertex, the results revealed a geometric regularity effect: detection was considerably easier when either the base shape or the intruder was a regular figure comprising right angles, parallelism, or symmetry rather than a more irregular shape. This effect was replicated in several tasks and in all human populations tested, including uneducated Himba adults and French kindergartners. Baboons, however, showed no such geometric regularity effect, even after extensive training. Baboon behavior was captured by convolutional neural networks (CNNs), but neither CNNs nor a variational autoencoder captured the human geometric regularity effect. However, a symbolic model, based on exact properties of Euclidean geometry, closely fitted human behavior. Our results indicate that the human propensity for symbolic abstraction permeates even elementary shape perception. They suggest a putative signature of human singularity and provide a challenge for nonsymbolic models of human shape perception.

Rotational-symmetry in a 3D scene and its 2D image

A 3D shape of an object is N-fold rotational-symmetric if the shape is invariant for 360/N degree rotations about an axis. Human observers are sensitive to the 2D rotational-symmetry of a retinal image, but they are less sensitive than they are to 2D mirror-symmetry, which involves invariance to reflection across an axis. Note that perception of the mirror-symmetry of a 2D image and a 3D shape has been well studied, where it has been shown that observers are sensitive to the mirror-symmetry of a 3D shape, and that 3D mirror-symmetry plays a critical role in the veridical perception of a 3D shape from its 2D image. On the other hand, the perception of rotational-symmetry, especially 3D rotational-symmetry, has received very little study. In this paper, we derive the geometrical properties of 2D and 3D rotational-symmetry and compare them to the geometrical properties of mirror-symmetry. Then, we discuss perceptual differences between mirror- and rotational symmetry based on this comparison. We found that rotational-symmetry has many geometrical properties that are similar to the geometrical properties of mirror-symmetry, but note that the 2D projection of a 3D rotational-symmetrical shape is more complex computationally than the 2D projection of a 3D mirror-symmetrical shape. This computational difficulty could make the human visual system less sensitive to the rotational-symmetry of a 3D shape than its mirror-symmetry.

Bayesian validation of grammar productions for the language of thought

Probabilistic proposals of Language of Thoughts (LoTs) can explain learning across different domains as statistical inference over a compositionally structured hypothesis space. While frameworks may differ on how a LoT may be implemented computationally, they all share the property that they are built from a set of atomic symbols and rules by which these symbols can be combined. In this work we propose an extra validation step for the set of atomic productions defined by the experimenter. It starts by expanding the defined LoT grammar for the cognitive domain with a broader set of arbitrary productions and then uses Bayesian inference to prune the productions from the experimental data. The result allows the researcher to validate that the resulting grammar still matches the intuitive grammar chosen for the domain. We then test this method in the language of geometry, a specific LoT model for geometrical sequence learning. Finally, despite the fact of the geometrical LoT not being a universal (i.e. Turing-complete) language, we show an empirical relation between a sequence’s probability and its complexity consistent with the theoretical relationship for universal languages described by Levin’s Coding Theorem.

Production of Supra-regular Spatial Sequences by Macaque Monkeys

Understanding and producing embedded sequences in language, music, or mathematics, is a central characteristic of our species. These domains are hypothesized to involve a human-specific competence for supra-regular grammars, which can generate embedded sequences that go beyond the regular sequences engendered by finite-state automata. However, is this capacity truly unique to humans? Using a production task, we show that macaque monkeys can be trained to produce time-symmetrical embedded spatial sequences whose formal description requires supra-regular grammars or, equivalently, a push-down stack automaton. Monkeys spontaneously generalized the learned grammar to novel sequences, including longer ones, and could generate hierarchical sequences formed by an embedding of two levels of abstract rules. Compared to monkeys, however, preschool children learned the grammars much faster using a chunking strategy. While supra-regular grammars are accessible to nonhuman primates through extensive training, human uniqueness may lie in the speed and learning strategy with which they are acquired.

Multiple processes in two-dimensional visual statistical learning

Knowledge about the arrangement of visual elements is an important aspect of perception. This study investigates whether humans learn rules of two-dimensional abstract patterns (exemplars) generated from Reber's artificial grammar. The key question is whether the subjects can implicitly learn them without explicit instructions, and, if so, how they use the acquired knowledge to judge new patterns (probes) in relation to their finite experience of the exemplars. The analysis was conducted using dissimilarities among patterns, which are defined with n-gram probabilities and the Levenshtein distance. The results show that subjects are able to learn rules of two-dimensional visual patterns (exemplars) and make categorical judgment of probes based on knowledge of exemplar-based representation. Our analysis revealed that subjects' judgments of probes were related to the degree of dissimilarities between the probes and exemplars. The result suggests the coexistence of configural and element-based processing in exemplar-based representations. Exemplar-based representation was preferred to prototypical representation through tasks requiring discrimination, recognition and working memory. Relations of the studied judgment processes to the neural basis are discussed. We conclude that knowledge of a finite experience of two-dimensional visual patterns would be crystalized in different levels of relations among visual elements.

The language of geometry: Fast comprehension of geometrical primitives and rules in human adults and preschoolers

During language processing, humans form complex embedded representations from sequential inputs. Here, we ask whether a "geometrical language" with recursive embedding also underlies the human ability to encode sequences of spatial locations. We introduce a novel paradigm in which subjects are exposed to a sequence of spatial locations on an octagon, and are asked to predict future locations. The sequences vary in complexity according to a well-defined language comprising elementary primitives and recursive rules. A detailed analysis of error patterns indicates that primitives of symmetry and rotation are spontaneously detected and used by adults, preschoolers, and adult members of an indigene group in the Amazon, the Munduruku, who have a restricted numerical and geometrical lexicon and limited access to schooling. Furthermore, subjects readily combine these geometrical primitives into hierarchically organized expressions. By evaluating a large set of such combinations, we obtained a first view of the language needed to account for the representation of visuospatial sequences in humans, and conclude that they encode visuospatial sequences by minimizing the complexity of the structured expressions that capture them.

Fluency Expresses Implicit Knowledge of Tonal Symmetry

The purposes of the present study were twofold. First, we sought to establish whether tonal symmetry produces processing fluency. Second, we sought to explore whether symmetry and chunk strength express themselves differently in fluency, as an indication of different mechanisms being involved for sub- and supra-finite state processing. Across two experiments, participants were asked to listen to and memorize artificial poetry showing a mirror symmetry (an inversion, i.e., a type of cross serial dependency); after this training phase, people completed a four-choice RT task in which they were presented with new artificial poetry. Participants were required to identify the stimulus displayed. We found that symmetry sped up responding to the second half of strings, indicating a fluency effect. Furthermore, there was a dissociation between fluency effects arising from symmetry vs. chunk strength, with stronger fluency effects for symmetry rather than chunks in the second half of strings. Taken together, we conjecture a divide between finite state and supra-finite state mechanisms in learning grammatical sequences.

More than one way to see it: Individual heuristics in avian visual computation

Comparative pattern learning experiments investigate how different species find regularities in sensory input, providing insights into cognitive processing in humans and other animals. Past research has focused either on one species' ability to process pattern classes or different species' performance in recognizing the same pattern, with little attention to individual and species-specific heuristics and decision strategies. We trained and tested two bird species, pigeons (Columba livia) and kea (Nestor notabilis, a parrot species), on visual patterns using touch-screen technology. Patterns were composed of several abstract elements and had varying degrees of structural complexity. We developed a model selection paradigm, based on regular expressions, that allowed us to reconstruct the specific decision strategies and cognitive heuristics adopted by a given individual in our task. Individual birds showed considerable differences in the number, type and heterogeneity of heuristic strategies adopted. Birds' choices also exhibited consistent species-level differences. Kea adopted effective heuristic strategies, based on matching learned bigrams to stimulus edges. Individual pigeons, in contrast, adopted an idiosyncratic mix of strategies that included local transition probabilities and global string similarity. Although performance was above chance and quite high for kea, no individual of either species provided clear evidence of learning exactly the rule used to generate the training stimuli. Our results show that similar behavioral outcomes can be achieved using dramatically different strategies and highlight the dangers of combining multiple individuals in a group analysis. These findings, and our general approach, have implications for the design of future pattern learning experiments, and the interpretation of comparative cognition research more generally.

Non-adjacent visual dependency learning in chimpanzees

Humans have a strong proclivity for structuring and patterning stimuli: Whether in space or time, we tend to mentally order stimuli in our environment and organize them into units with specific types of relationships. A crucial prerequisite for such organization is the cognitive ability to discern and process regularities among multiple stimuli. To investigate the evolutionary roots of this cognitive capacity, we tested chimpanzees—which, along with bonobos, are our closest living relatives—for simple, variable distance dependency processing in visual patterns. We trained chimpanzees to identify pairs of shapes either linked by an arbitrary learned association (arbitrary associative dependency) or a shared feature (same shape, feature-based dependency), and to recognize strings where items related to either of these ways occupied the first (leftmost) and the last (rightmost) item of the stimulus. We then probed the degree to which subjects generalized this pattern to new colors, shapes, and numbers of interspersed items. We found that chimpanzees can learn and generalize both types of dependency rules, indicating that the ability to encode both feature-based and arbitrary associative regularities over variable distances in the visual domain is not a human prerogative. Our results strongly suggest that these core components of human structural processing were already present in our last common ancestor with chimpanzees.

Toward a computational framework for cognitive biology: unifying approaches from cognitive neuroscience and comparative cognition

Progress in understanding cognition requires a quantitative, theoretical framework, grounded in the other natural sciences and able to bridge between implementational, algorithmic and computational levels of explanation. I review recent results in neuroscience and cognitive biology that, when combined, provide key components of such an improved conceptual framework for contemporary cognitive science. Starting at the neuronal level, I first discuss the contemporary realization that single neurons are powerful tree-shaped computers, which implies a reorientation of computational models of learning and plasticity to a lower, cellular, level. I then turn to predictive systems theory (predictive coding and prediction-based learning) which provides a powerful formal framework for understanding brain function at a more global level. Although most formal models concerning predictive coding are framed in associationist terms, I argue that modern data necessitate a reinterpretation of such models in cognitive terms: as model-based predictive systems. Finally, I review the role of the theory of computation and formal language theory in the recent explosion of comparative biological research attempting to isolate and explore how different species differ in their cognitive capacities. Experiments to date strongly suggest that there is an important difference between humans and most other species, best characterized cognitively as a propensity by our species to infer tree structures from sequential data. Computationally, this capacity entails generative capacities above the regular (finite-state) level; implementationally, it requires some neural equivalent of a push-down stack. I dub this unusual human propensity "dendrophilia", and make a number of concrete suggestions about how such a system may be implemented in the human brain, about how and why it evolved, and what this implies for models of language acquisition. I conclude that, although much remains to be done, a neurally-grounded framework for theoretical cognitive science is within reach that can move beyond polarized debates and provide a more adequate theoretical future for cognitive biology.

Spatial analysis of "crazy quilts", a class of potentially random aesthetic artefacts

Human artefacts in general are highly structured and often display ordering principles such as translational, reflectional or rotational symmetry. In contrast, human artefacts that are intended to appear random and non symmetrical are very rare. Furthermore, many studies show that humans find it extremely difficult to recognize or reproduce truly random patterns or sequences. Here, we attempt to model two-dimensional decorative spatial patterns produced by humans that show no obvious order. “Crazy quilts” represent a historically important style of quilt making that became popular in the 1870s, and lasted about 50 years. Crazy quilts are unusual because unlike most human artefacts, they are specifically intended to appear haphazard and unstructured. We evaluate the degree to which this intention was achieved by using statistical techniques of spatial point pattern analysis to compare crazy quilts with regular quilts from the same region and era and to evaluate the fit of various random distributions to these two quilt classes. We found that the two quilt categories exhibit fundamentally different spatial characteristics: The patch areas of crazy quilts derive from a continuous random distribution, while area distributions of regular quilts consist of Gaussian mixtures. These Gaussian mixtures derive from regular pattern motifs that are repeated and we suggest that such a mixture is a distinctive signature of human-made visual patterns. In contrast, the distribution found in crazy quilts is shared with many other naturally occurring spatial patterns. Centroids of patches in the two quilt classes are spaced differently and in general, crazy quilts but not regular quilts are well-fitted by a random Strauss process. These results indicate that, within the constraints of the quilt format, Victorian quilters indeed achieved their goal of generating random structures.

The nature of the memory buffer in implicit learning: learning Chinese tonal symmetries

Previous research has established that people can implicitly learn chunks, which (in terms of formal language theory) do not require a memory buffer to process. The present study explores the implicit learning of nonlocal dependencies generated by higher than finite-state grammars, specifically, Chinese tonal retrogrades (i.e. centre embeddings generated from a context-free grammar) and inversions (i.e. cross-serial dependencies generated from a mildly context-sensitive grammar), which do require buffers (for example, last in-first out and first in-first out, respectively). People were asked to listen to and memorize artificial poetry instantiating one of the two grammars; after this training phase, people were informed of the existence of rules and asked to classify new poems, while providing attributions of the basis of their judgments. People acquired unconscious structural knowledge of both tonal retrogrades and inversions. Moreover, inversions were implicitly learnt more easily than retrogrades constraining the nature of the memory buffer in computational models of implicit learning.

Fechner revisited: towards an inclusive approach to aesthetics

Accepting Bullot & Reber's (B&R's) criteria for art appreciation would confine the study of aesthetics to those works for which historical information is available, mainly post-eighteenth-century Western "high art." We reject their contention that "correct" artistic understanding is limited to experts with detailed knowledge or education in art, which implies a narrowly elitist conception of aesthetics. Scientific aesthetics must be broadly inclusive.

Pattern perception and computational complexity: introduction to the special issue

Research on pattern perception and rule learning, grounded in formal language theory (FLT) and using artificial grammar learning paradigms, has exploded in the last decade. This approach marries empirical research conducted by neuroscientists, psychologists and ethologists with the theory of computation and FLT, developed by mathematicians, linguists and computer scientists over the last century. Of particular current interest are comparative extensions of this work to non-human animals, and neuroscientific investigations using brain imaging techniques. We provide a short introduction to the history of these fields, and to some of the dominant hypotheses, to help contextualize these ongoing research programmes, and finally briefly introduce the papers in the current issue.

Artificial grammar learning meets formal language theory: an overview

Formal language theory (FLT), part of the broader mathematical theory of computation, provides a systematic terminology and set of conventions for describing rules and the structures they generate, along with a rich body of discoveries and theorems concerning generative rule systems. Despite its name, FLT is not limited to human language, but is equally applicable to computer programs, music, visual patterns, animal vocalizations, RNA structure and even dance. In the last decade, this theory has been profitably used to frame hypotheses and to design brain imaging and animal-learning experiments, mostly using the 'artificial grammar-learning' paradigm. We offer a brief, non-technical introduction to FLT and then a more detailed analysis of empirical research based on this theory. We suggest that progress has been hampered by a pervasive conflation of distinct issues, including hierarchy, dependency, complexity and recursion. We offer clarifications of several relevant hypotheses and the experimental designs necessary to test them. We finally review the recent brain imaging literature, using formal languages, identifying areas of convergence and outstanding debates. We conclude that FLT has much to offer scientists who are interested in rigorous empirical investigations of human cognition from a neuroscientific and comparative perspective.