Lateralization of kin recognition signals in the human face

Facial Kinship Verification: A Comprehensive Review and Outlook

The goal of Facial Kinship Verification (FKV) is to automatically determine whether two individuals have a kin relationship or not from their given facial images or videos. It is an emerging and challenging problem that has attracted increasing attention due to its practical applications. Over the past decade, significant progress has been achieved in this new field. Handcrafted features and deep learning techniques have been widely studied in FKV. The goal of this paper is to conduct a comprehensive review of the problem of FKV. We cover different aspects of the research, including problem definition, challenges, applications, benchmark datasets, a taxonomy of existing methods, and state-of-the-art performance. In retrospect of what has been achieved so far, we identify gaps in current research and discuss potential future research directions.

Reasoning Graph Networks for Kinship Verification: From Star-Shaped to Hierarchical

In this paper, we investigate the problem of facial kinship verification by learning hierarchical reasoning graph networks. Conventional methods usually focus on learning discriminative features for each facial image of a paired sample and neglect how to fuse the obtained two facial image features and reason about the relations between them. To address this, we propose a Star-shaped Reasoning Graph Network (S-RGN). Our S-RGN first constructs a star-shaped graph where each surrounding node encodes the information of comparisons in a feature dimension and the central node is employed as the bridge for the interaction of surrounding nodes. Then we perform relational reasoning on this star graph with iterative message passing. The proposed S-RGN uses only one central node to analyze and process information from all surrounding nodes, which limits its reasoning capacity. We further develop a Hierarchical Reasoning Graph Network (H-RGN) to exploit more powerful and flexible capacity. More specifically, our H-RGN introduces a set of latent reasoning nodes and constructs a hierarchical graph with them. Then bottom-up comparative information abstraction and top-down comprehensive signal propagation are iteratively performed on the hierarchical graph to update the node features. Extensive experimental results on four widely used kinship databases show that the proposed methods achieve very competitive results.

Weighted Graph Embedding-Based Metric Learning for Kinship Verification

Given a group photograph, it is interesting and useful to judge whether the characters in it share specific kinship relation, such as father-daughter, father-son, mother-daughter, or mother-son. Recently, facial image-based kinship verification has attracted wide attention in computer vision. Some metric learning algorithms have been developed for improving kinship verification. However, most of the existing algorithms ignore fusing multiple feature representations and utilizing kernel techniques. In this paper, we develop a novel weighted graph embedding-based metric learning (WGEML) framework for kinship verification. Inspired by the fact that family members usually show high similarity in facial features like eyes, noses, and mouths, despite their diversity, we jointly learn multiple metrics by constructing an intrinsic graph and two penalty graphs to characterize the intraclass compactness and interclass separability for each feature representation, respectively, so that both the consistency and complementarity among multiple features can be fully exploited. Meanwhile, combination weights are determined through a weighted graph embedding framework. Furthermore, we present a kernelized version of WGEML to tackle nonlinear problems. Experimental results demonstrate both the effectiveness and efficiency of our proposed methods.

Birth Order Does Not Affect Ability to Detect Kin

Previous studies suggest that birth order affects kinship detection ability. Kaminski et al. (2010) argued that firstborns use contextual cues (e.g., maternal perinatal association) to assess kinship in their own family, leading to a disadvantage in assessing kinship from facial cues alone in strangers. In contrast, laterborns do not have the contextual cue of maternal perinatal association and hence rely more on facial cues, leading to an advantage in detecting kin from facial cues alone. However, Alvergne et al. (2010) found no evidence in support of such a birthorder effect. The current study aimed to replicate previous studies with better suited methods to determine the effect of birth order on kin recognition. 109 raters viewed 132 pairs of photographs of children (aged 3–17 years), and indicated whether each pair was related or unrelated. Half of the pairs were sibling pairs and half were unrelated child pairs that were age- and gender- matched to the related pairs. No image was shown more than once, related pairs were not known to be related to any other image in the study, and individuals from unrelated pairs were not known to be related to any other image. We used binomial logistic mixed effects modelling to predict kinship judgments from relatedness and birth order (with image pair and rater as random factors). Relatedness was the main factor driving kinship judgments; related child-pairs were more than twice as likely as unrelated pairs to be judged as kin. Kinship judgment accuracy was unaffected by rater birth order. These findings indicate that laterborns did not have an advantage in detecting child sibling pairs. Pre-registration, data, code, and preprint available at osf.io/h43ep.

Triangular Similarities of Facial Features to Determine: The Relationships Among Family Members

An image can be represented in the form of patterns of intensities, with the objects of an image appearing in the form of a pattern on an X-Y plane. The two patterns of intensities of two corresponding facial images are measured by calculating the areas of right triangles formed from patterns in a Cartesian coordinate system. The purpose of representing patterns of intensities in the Cartesian coordinate system is to measure the percentage of similarities that exists between two facial images, similarities inherent in photographs. The percentage is measured by incorporating the proposed technique of areas that are common between two patterns of intensities. The pattern 1 produces areas of right triangles of a parent with respect to areas of right triangles of a child. The strategy of measuring the facial similarities between two patterns of intensities is dependent on the areas of pattern 1 that have commonalities with the areas of pattern 2. This helps in the measuring of the facial similarities between two patterns of intensities. The proposed method has yielded results of 71.3, 77.1, 71.3, and 70.5 percent of similarity on the dataset KinfaceW-I and 80.7, 82.1, 80.6, 81.1 on the dataset KinfaceW-II.

Hierarchical Representation Learning for Kinship Verification

Kinship verification has a number of applications such as organizing large collections of images and recognizing resemblances among humans. In this paper, first, a human study is conducted to understand the capabilities of human mind and to identify the discriminatory areas of a face that facilitate kinship-cues. The visual stimuli presented to the participants determine their ability to recognize kin relationship using the whole face as well as specific facial regions. The effect of participant gender and age and kin-relation pair of the stimulus is analyzed using quantitative measures such as accuracy, discriminability index d' , and perceptual information entropy. Utilizing the information obtained from the human study, a hierarchical kinship verification via representation learning (KVRL) framework is utilized to learn the representation of different face regions in an unsupervised manner. We propose a novel approach for feature representation termed as filtered contractive deep belief networks (fcDBN). The proposed feature representation encodes relational information present in images using filters and contractive regularization penalty. A compact representation of facial images of kin is extracted as an output from the learned model and a multi-layer neural network is utilized to verify the kin accurately. A new WVU kinship database is created, which consists of multiple images per subject to facilitate kinship verification. The results show that the proposed deep learning framework (KVRL-fcDBN) yields the state-of-the-art kinship verification accuracy on the WVU kinship database and on four existing benchmark data sets. Furthermore, kinship information is used as a soft biometric modality to boost the performance of face verification via product of likelihood ratio and support vector machine based approaches. Using the proposed KVRL-fcDBN framework, an improvement of over 20% is observed in the performance of face verification.

Allocentric kin recognition is not affected by facial inversion

Typical judgments involving faces are disrupted by inversion, with the Thatcher illusion serving as a compelling example. In two experiments, we examined how inversion affects allocentric kin recognition-the ability to judge the degree of genetic relatedness of others. In the first experiment, participants judged whether pairs of photographs of children portrayed siblings or unrelated children. Half of the pairs were siblings, half were unrelated. In three experimental conditions, photographs were viewed in upright orientation, flipped around a horizontal axis, or rotated 180°. Neither rotation nor flipping had any detectable effect on allocentric kin recognition. In the second experiment, participants judged pairs of photographs of adult women. Half of the pairs were sisters, half were unrelated. We again found no significant effect of facial inversion. Unlike almost all other face judgments, judgments of kinship from facial appearance do not rely on perceptual cues disrupted by inversion, suggesting that they rely more on spatially localized cues rather than "holistic" cues. We conclude that kin recognition is not simply a byproduct of other face perception abilities. We discuss the implications for cue combination models of other facial judgments that are affected by inversion.

Identification of visual paternity cues in humans

Understanding how individuals identify their relatives has implications for the evolution of social behaviour. Kinship cues might be based on familiarity, but in the face of paternity uncertainty and costly paternal investment, other mechanisms such as phenotypic matching may have evolved. In humans, paternal recognition of offspring and subsequent discriminative paternal investment have been linked to father-offspring facial phenotypic similarities. However, the extent to which paternity detection is impaired by environmentally induced facial information is unclear. We used 27 portraits of fathers and their adult sons to quantify the level of paternity detection according to experimental treatments that manipulate the location, type and quantity of visible facial information. We found that (i) the lower part of the face, that changes most with development, does not contain paternity cues, (ii) paternity can be detected even if relational information within the face is disrupted and (iii) the signal depends on the presence of specific information rather than their number. Taken together, the results support the view that environmental effects have little influence on the detection of paternity using facial similarities. This suggests that the cognitive dispositions enabling the facial detection of kinship relationships ignore genetic irrelevant facial information.

Neighborhood repulsed metric learning for kinship verification

Kinship verification from facial images is an interesting and challenging problem in computer vision, and there are very limited attempts on tackle this problem in the literature. In this paper, we propose a new neighborhood repulsed metric learning (NRML) method for kinship verification. Motivated by the fact that interclass samples (without a kinship relation) with higher similarity usually lie in a neighborhood and are more easily misclassified than those with lower similarity, we aim to learn a distance metric under which the intraclass samples (with a kinship relation) are pulled as close as possible and interclass samples lying in a neighborhood are repulsed and pushed away as far as possible, simultaneously, such that more discriminative information can be exploited for verification. To make better use of multiple feature descriptors to extract complementary information, we further propose a multiview NRML (MNRML) method to seek a common distance metric to perform multiple feature fusion to improve the kinship verification performance. Experimental results are presented to demonstrate the efficacy of our proposed methods. Finally, we also test human ability in kinship verification from facial images and our experimental results show that our methods are comparable to that of human observers.