Assessing the potential information content of multicomponent visual signals: a machine learning approach

Evolution of Plumage Patterns in a Pattern Morphospace: A Phylogenetic Analysis of Melanerpine Woodpeckers

AbstractPlumage patterns of melanerpine (Melanerpes-Sphyrapicus) woodpeckers are strikingly diverse. Understanding the evolution and function of this diversity is challenging because of the difficulty of quantifying plumage patterns. We use a three-dimensional space to characterize the evolution of melanerpine achromatic plumage patterns. The axes of the space are three pattern features (spatial frequency, orientation, and contrast) quantified using two-dimensional fast Fourier transformation of museum specimen images. Mapping plumage in pattern space reveals differences in how species and subclades occupy the space. To quantify these differences, we derive two new measures of pattern: pattern diversity (diversity across plumage patches within a species) and pattern uniqueness (divergence of patterns from those of other species). We estimate that the melanerpine ancestor had mottled plumage and find that pattern traits across patches and subclades evolve at different rates. We also find that smaller species are more likely to display horizontal face patterning. We promote pattern spaces as powerful tools for investigating animal pattern evolution.

Revisiting the baby schema by a geometric morphometric analysis of infant facial characteristics across great apes

Infants across species are thought to exhibit specific facial features (termed the "baby schema", such as a relatively bigger forehead and eyes, and protruding cheeks), with an adaptive function to induce caretaking behaviour from adults. There is abundant empirical evidence for this in humans, but, surprisingly, the existence of a baby schema in non-human animals has not been scientifically demonstrated. We investigated which facial characteristics are shared across infants in five species of great apes: humans, chimpanzees, bonobos, mountain gorillas, and Bornean orangutans. We analysed eight adult and infant faces for each species (80 images in total) using geometric morphometric analysis and machine learning. We found two principal components characterizing infant faces consistently observed across species. These included (1) relatively bigger eyes located lower in the face, (2) a rounder and vertically shorter face shape, and (3) an inverted triangular face shape. While these features are shared, human infant faces are unique in that the second characteristic (round face shape) is more pronounced, whereas the third (inverted triangular face shape) is less pronounced than other species. We also found some infantile features only found in some species. We discuss future directions to investigate the baby schema using an evolutionary approach.

Simulated evolution of mating signal diversification in a primate radiation

Divergence in allopatry and subsequent diversification of mating signals on secondary contact (reinforcement) is a major driver of phenotypic diversity. Observing this evolutionary process directly is often impossible, but simulated evolution can pinpoint key drivers of phenotypic variation. We developed evolutionary simulations in which mating signals, modelled as points in phenotype space, evolve across time under varying evolutionary scenarios. We model mate recognition signals in guenons, a primate radiation exhibiting colourful and diverse face patterns hypothesized to maintain reproductive isolation via mate choice. We simulate face pattern evolution across periods of allopatry and sympatry, identifying the role of key parameters in driving evolutionary endpoints. Results show that diversification in allopatry and assortative mate choice on secondary contact can induce rapid phenotypic diversification, resulting in distinctive (between species) and stereotyped (within species) face patterns, similar to extant guenons. Strong selection against hybrids is key to diversification, with even low levels of hybrid fitness often resulting in merged populations on secondary contact. Our results support a key role for reinforcement by assortative mating in the maintenance of species diversity and support the long-proposed prehistorical scenario for how such striking diversity was produced and maintained in perhaps the most colourful of all mammalian clades.

Perspectives on Individual Animal Identification from Biology and Computer Vision

Identifying individual animals is crucial for many biological investigations. In response to some of the limitations of current identification methods, new automated computer vision approaches have emerged with strong performance. Here, we review current advances of computer vision identification techniques to provide both computer scientists and biologists with an overview of the available tools and discuss their applications. We conclude by offering recommendations for starting an animal identification project, illustrate current limitations, and propose how they might be addressed in the future.

Using comprehensive machine-learning models to classify complex morphological characters

Recognizing and classifying multiple morphological features, such as patterns, sizes, and textures, can provide a comprehensive understanding of their variability and phenotypic evolution. Yet, quantitatively measuring complex morphological characters remains challenging.We provide a machine learning-based pipeline (SVMorph) to consider and classify complex morphological characters in multiple organisms that have either small or large datasets.Our pipeline integrates two descriptors, histogram of oriented gradient and local binary pattern, to meet various classification needs. We also optimized feature extraction by adding image data augmentation to improve model generalizability.We tested SVMorph on two real-world examples to demonstrate that it can be used with small training datasets and limited computational resources. Comparing with multiple CNN-based methods and traditional techniques, we show that SVMorph is reliable and fast in texture-based individual classification. Thus, SVMorph can be used to efficiently classify multiple morphological characters in distinct nonmodel organisms.

Identifying female phenotypes that promote behavioral isolation in a sexually dimorphic species of fish Etheostoma zonale

In sexually dimorphic species characterized by exaggerated male ornamentation, behavioral isolation is often attributed to female preferences for conspecific male signals. Yet, in a number of sexually dimorphic species, male mate choice also results in behavioral isolation. In many of these cases, the female traits that mediate species boundaries are unclear. Females in sexually dimorphic species typically lack many of the elaborate traits that are present in males and that are often used for taxonomic classification of species. In a diverse and largely sexually dimorphic group of fishes called darters (Percidae: Etheostoma), male mate choice contributes to behavioral isolation between a number of species; however, studies addressing which female traits males prefer are lacking. In this study, we identified the dominant female pattern for two sympatric species, Etheostoma zonale and Etheostoma barrenense, using pattern energy analysis, and we used discriminate function analysis to identify which aspects of female patterning can reliably classify species. We then tested the role of female features in male mate choice for E. zonale, by measuring male preference for computer animations displaying the identified (species-specific) conspecific features. We found that the region above the lateral line is important in mediating male mate preferences, with males spending a significantly greater proportion of time with animations exhibiting conspecific female patterning in this region than with animations exhibiting heterospecific female patterning. Our results suggest that the aspects of female phenotypes that are the target of male mate choice are different from the conspicuous male phenotypes that traditionally characterize species.

Domestication Phenotype Linked to Vocal Behavior in Marmoset Monkeys

The domestication syndrome refers to a set of traits that are the by-products of artificial selection for increased tolerance toward humans [1-3]. One hypothesis is that some species, like humans and bonobos, "self-domesticated" and have been under selection for that same suite of domesticated phenotypes [4-8]. However, the evidence for this has been largely circumstantial. Here, we provide evidence that, in marmoset monkeys, the size of a domestication phenotype-a white facial fur patch-is linked to their degree of affiliative vocal responding. During development, the amount of parental vocal feedback experienced influences the rate of growth of this facial white patch, and this suggests a mechanistic link between the two phenotypes, possibly via neural crest cells. Our study provides evidence for links between vocal behavior and the development of morphological phenotypes associated with domestication in a nonhuman primate.

Coevolution of cognitive abilities and identity signals in individual recognition systems

Social interactions are mediated by recognition systems, meaning that the cognitive abilities or phenotypic diversity that facilitate recognition may be common targets of social selection. Recognition occurs when a receiver compares the phenotypes produced by a sender with a template. Coevolution between sender and receiver traits has been empirically reported in multiple species and sensory modalities, though the dynamics and relative exaggeration of traits from senders versus receivers have received little attention. Here, we present a coevolutionary dynamic model that examines the conditions under which senders and receivers should invest effort in facilitating individual recognition. The model predicts coevolution of sender and receiver traits, with the equilibrium investment dependent on the relative costs of signal production versus cognition. In order for recognition to evolve, initial sender and receiver trait values must be above a threshold, suggesting that recognition requires some degree of pre-existing diversity and cognitive abilities. The analysis of selection gradients demonstrates that the strength of selection on sender signals and receiver cognition is strongest when the trait values are furthest from the optima. The model provides new insights into the expected strength and dynamics of selection during the origin and elaboration of individual recognition, an important feature of social cognition in many taxa. This article is part of the theme issue 'Signal detection theory in recognition systems: from evolving models to experimental tests'.

The structure of species discrimination signals across a primate radiation

Discriminating conspecifics from heterospecifics can help avoid costly interactions between closely related sympatric species. The guenons, a recent primate radiation, exhibit high degrees of sympatry and form multi-species groups. Guenons have species-specific colorful face patterns hypothesized to function in species discrimination. Here, we use a machine learning approach to identify face regions most essential for species classification across fifteen guenon species. We validate these computational results using experiments with live guenons, showing that facial traits critical for accurate classification influence selective attention toward con- and heterospecific faces. Our results suggest variability among guenon species in reliance on single-trait-based versus holistic facial characteristics for species discrimination, with behavioral responses and computational results indicating variation from single-trait to whole-face patterns. Our study supports a role for guenon face patterns in species discrimination, and shows how complex signals can be informative about differences between species across a speciose and highly sympatric radiation.

Seeing spots: quantifying mother-offspring similarity and assessing fitness consequences of coat pattern traits in a wild population of giraffes (Giraffa camelopardalis)

Polymorphic phenotypes of mammalian coat coloration have been important to the study of genetics and evolution, but less is known about the inheritance and fitness consequences of individual variation in complex coat pattern traits such as spots and stripes. Giraffe coat markings are highly complex and variable and it has been hypothesized that variation in coat patterns most likely affects fitness by camouflaging neonates against visually hunting predators. We quantified complex coat pattern traits of wild Masai giraffes using image analysis software, determined the similarity of spot pattern traits between mother and offspring, and assessed whether variation in spot pattern traits was related to fitness as measured by juvenile survival. The methods we described could comprise a framework for objective quantification of complex mammal coat pattern traits based on photographic coat pattern data. We demonstrated that some characteristics of giraffe coat spot shape were likely to be heritable, as measured by mother-offspring regression. We found significant variation in juvenile survival among phenotypic groups of neonates defined by multivariate clustering based on spot trait measurement variables. We also found significant variation in neonatal survival associated with spot size and shape covariates. Larger spots (smaller number of spots) and irregularly shaped or rounder spots (smaller aspect ratio) were correlated with increased survival. These findings will inform investigations into developmental and genetic architecture of complex mammal coat patterns and their adaptive value.

Signals from the brain and olfactory epithelium control shaping of the mammalian nasal capsule cartilage

Facial shape is the basis for facial recognition and categorization. Facial features reflect the underlying geometry of the skeletal structures. Here, we reveal that cartilaginous nasal capsule (corresponding to upper jaw and face) is shaped by signals generated by neural structures: brain and olfactory epithelium. Brain-derived Sonic Hedgehog (SHH) enables the induction of nasal septum and posterior nasal capsule, whereas the formation of a capsule roof is controlled by signals from the olfactory epithelium. Unexpectedly, the cartilage of the nasal capsule turned out to be important for shaping membranous facial bones during development. This suggests that conserved neurosensory structures could benefit from protection and have evolved signals inducing cranial cartilages encasing them. Experiments with mutant mice revealed that the genomic regulatory regions controlling production of SHH in the nervous system contribute to facial cartilage morphogenesis, which might be a mechanism responsible for the adaptive evolution of animal faces and snouts.

The role of facial pattern variation for species recognition in red-fronted lemurs (Eulemur rufifrons)

BACKGROUND

Species recognition, i.e., the ability to distinguish conspecifics from heterospecifics, plays an essential role in reproduction. The role of facial cues for species recognition has been investigated in several non-human primate species except for lemurs. We therefore investigated the role of facial cues for species recognition in wild red-fronted lemurs (Eulemur rufifrons) at Kirindy Forest. We presented adult red-fronted lemurs pictures of male faces from five species including red-fronted lemurs, three closely related species, white-fronted lemurs (E. albifrons), brown lemurs (E. fulvus), rufous brown lemurs (E. rufus), and genetically more distant red-bellied lemurs (E. rubriventer), occurring in allopatry with the study population. We predicted that red-fronted lemurs respond stronger to conspecific than to heterospecific pictures and that females show stronger responses than males. In addition, if genetic drift has played a role in the evolution of facial color patterns in the members of this genus, we predicted that responses of red-fronted lemurs correlate negatively with the genetic distance to the different species stimuli.

RESULTS

Red-fronted lemurs looked significantly longer at pictures of their own species than at those of heterospecifics. Females spent less time looking at pictures of white-fronted, brown and red-bellied lemurs than males did, but not to pictures of red-bellied lemurs and a control stimulus. Individuals also exhibited sniffing behavior while looking at visual stimuli, and the time spent sniffing was significantly longer for pictures of conspecifics compared to those of heterospecifics. Moreover, the time spent looking and sniffing towards the pictures correlated negatively with the genetic distance between their own species and the species presented as stimulus.

CONCLUSIONS

We conclude that red-fronted lemurs have the ability for species recognition using visual facial cues, which may allow them to avoid costly interbreeding. If so, sexual selection might have influenced the evolution of facial patterns in eulemurs. Since responses also correlated with genetic distance, our findings suggest a potential role of genetic drift as well as sexual selection in influencing the evolution of facial variation in eulemurs. Because study subjects looked and sniffed towards the presented pictures, red-fronted lemurs might have the ability for multi-modal species recognition.

Evolution of facial color pattern complexity in lemurs

Interspecific variation in facial color patterns across New and Old World primates has been linked to species recognition and group size. Because group size has opposite effects on interspecific variation in facial color patterns in these two radiations, a study of the third large primate radiation may shed light on convergences and divergences in this context. We therefore compiled published social and ecological data and analyzed facial photographs of 65 lemur species to categorize variation in hair length, hair and skin coloration as well as color brightness. Phylogenetically controlled analyses revealed that group size and the number of sympatric species did not influence the evolution of facial color complexity in lemurs. Climatic factors, however, influenced facial color complexity, pigmentation and hair length in a few facial regions. Hair length in two facial regions was also correlated with group size and may facilitate individual recognition. Since phylogenetic signals were moderate to high for most models, genetic drift may have also played a role in the evolution of facial color patterns of lemurs. In conclusion, social factors seem to have played only a subordinate role in the evolution of facial color complexity in lemurs, and, more generally, group size appears to have no systematic functional effect on facial color complexity across all primates.

Automated face recognition of rhesus macaques

BACKGROUND

Rhesus macaques are widely used in biomedical research. Automated behavior monitoring can be useful in various fields (including neuroscience), as well as having applications to animal welfare but current technology lags behind that developed for other species. One difficulty facing developers is the reliable identification of individual macaques within a group especially as pair- and group-housing of macaques becomes standard. Current published methods require either implantation or wearing of a tracking device.

NEW METHOD

I present face recognition, in combination with face detection, as a method to non-invasively identify individual rhesus macaques in videos. The face recognition method utilizes local-binary patterns in combination with a local discriminant classification algorithm.

RESULTS

A classification accuracy of between 90 and 96% was achieved for four different groups. Group size, number of training images and challenging image conditions such as high contrast all had an impact on classification accuracy. I demonstrate that these methods can be applied in real time using standard affordable hardware and a potential application to studies of social structure.

COMPARISON WITH EXISTING METHOD(S)

Face recognition methods have been reported for humans and other primate species such as chimpanzees but not rhesus macaques. The classification accuracy with this method is comparable to that for chimpanzees. Face recognition has the advantage over other methods for identifying rhesus macaques such as tags and collars of being non-invasive.

CONCLUSIONS

This is the first reported method for face recognition of rhesus macaques, has high classification accuracy and can be implemented in real time.

patternize: An R package for quantifying colour pattern variation

The use of image data to quantify, study and compare variation in the colors and patterns of organisms requires the alignment of images to establish homology, followed by color-based segmentation of images. Here we describe an R package for image alignment and segmentation that has applications to quantify color patterns in a wide range of organisms. patternize is an R package that quantifies variation in color patterns obtained from image data. patternize first defines homology between pattern positions across specimens either through manually placed homologous landmarks or automated image registration. Pattern identification is performed by categorizing the distribution of colors using an RGB threshold, k-means clustering or watershed transformation.We demonstrate that patternize can be used for quantification of the color patterns in a variety of organisms by analyzing image data for butterflies, guppies, spiders and salamanders. Image data can be compared between sets of specimens, visualized as heatmaps and analyzed using principal component analysis (PCA). patternize has potential applications for fine scale quantification of color pattern phenotypes in population comparisons, genetic association studies and investigating the basis of color pattern variation across a wide range of organisms.

Heritable variation in colour patterns mediating individual recognition

Understanding the developmental and evolutionary processes that generate and maintain variation in natural populations remains a major challenge for modern biology. Populations of Polistes fuscatus paper wasps have highly variable colour patterns that mediate individual recognition. Previous experimental and comparative studies have provided evidence that colour pattern diversity is the result of selection for individuals to advertise their identity. Distinctive identity-signalling phenotypes facilitate recognition, which reduces aggression between familiar individuals in P. fuscatus wasps. Selection for identity signals may increase phenotypic diversity via two distinct modes of selection that have different effects on genetic diversity. Directional selection for increased plasticity would greatly increase phenotypic diversity but decrease genetic diversity at associated loci. Alternatively, heritable identity signals under balancing selection would maintain genetic diversity at associated loci. Here, we assess whether there is heritable variation underlying colour pattern diversity used for facial recognition in a wild population of P. fuscatus wasps. We find that colour patterns are heritable and not Mendelian, suggesting that multiple loci are involved. Additionally, patterns of genetic correlations among traits indicated that many of the loci underlying colour pattern variation are unlinked and independently segregating. Our results support a model where the benefits of being recognizable maintain genetic variation at multiple unlinked loci that code for phenotypic diversity used for recognition.

Image calibration and analysis toolbox - a free software suite for objectively measuring reflectance, colour and pattern

Quantitative measurements of colour, pattern and morphology are vital to a growing range of disciplines. Digital cameras are readily available and already widely used for making these measurements, having numerous advantages over other techniques, such as spectrometry. However, off-the-shelf consumer cameras are designed to produce images for human viewing, meaning that their uncalibrated photographs cannot be used for making reliable, quantitative measurements. Many studies still fail to appreciate this, and of those scientists who are aware of such issues, many are hindered by a lack of usable tools for making objective measurements from photographs.We have developed an image processing toolbox that generates images that are linear with respect to radiance from the RAW files of numerous camera brands and can combine image channels from multispectral cameras, including additional ultraviolet photographs. Images are then normalised using one or more grey standards to control for lighting conditions. This enables objective measures of reflectance and colour using a wide range of consumer cameras. Furthermore, if the camera's spectral sensitivities are known, the software can convert images to correspond to the visual system (cone-catch values) of a wide range of animals, enabling human and non-human visual systems to be modelled. The toolbox also provides image analysis tools that can extract luminance (lightness), colour and pattern information. Furthermore, all processing is performed on 32-bit floating point images rather than commonly used 8-bit images. This increases precision and reduces the likelihood of data loss through rounding error or saturation of pixels, while also facilitating the measurement of objects with shiny or fluorescent properties.All cameras tested using this software were found to demonstrate a linear response within each image and across a range of exposure times. Cone-catch mapping functions were highly robust, converting images to several animal visual systems and yielding data that agreed closely with spectrometer-based estimates.Our imaging toolbox is freely available as an addition to the open source ImageJ software. We believe that it will considerably enhance the appropriate use of digital cameras across multiple areas of biology, in particular researchers aiming to quantify animal and plant visual signals.