Facial recognition technology can expose political orientation from naturalistic facial images

Smile variation leaks personality and increases the accuracy of interpersonal judgments

People ubiquitously smile during brief interactions and first encounters, and when posing for photos used for virtual dating, social networking, and professional profiles. Yet not all smiles are the same: subtle individual differences emerge in how people display this nonverbal facial expression. We hypothesized that idiosyncrasies in people's smiles can reveal aspects of their personality and guide the personality judgments made by observers, thus enabling a smiling face to serve as a valuable tool in making more precise inferences about an individual's personality. Study 1 (N = 303) supported the hypothesis that smile variation reveals personality, and identified the facial-muscle activations responsible for this leakage. Study 2 (N = 987) found that observers use the subtle distinctions in smiles to guide their personality judgments, consequently forming slightly more accurate judgments of smiling faces than neutral ones. Smiles thus encode traces of personality traits, which perceivers utilize as valid cues of those traits.

Predicting personality or prejudice? Facial inference in the age of artificial intelligence

Facial inference, a cornerstone of person perception, has traditionally been studied through human judgments about personality traits and abilities based on people's faces. Recent advances in artificial intelligence (AI) have introduced new dimensions to this field, employing machine learning algorithms to reveal people's character, capabilities, and social outcomes based just on their faces. This review examines recent research on human and AI-based facial inference across psychology, business, computer science, legal, and policy studies to highlight the need for scientific consensus on whether or not people's faces can reveal their inner traits, and urges researchers to address the critical concerns around epistemic validity, practical relevance, and societal welfare before recommending AI-based facial inference for consequential uses.

Py-Feat: Python Facial Expression Analysis Toolbox

Studying facial expressions is a notoriously difficult endeavor. Recent advances in the field of affective computing have yielded impressive progress in automatically detecting facial expressions from pictures and videos. However, much of this work has yet to be widely disseminated in social science domains such as psychology. Current state-of-the-art models require considerable domain expertise that is not traditionally incorporated into social science training programs. Furthermore, there is a notable absence of user-friendly and open-source software that provides a comprehensive set of tools and functions that support facial expression research. In this paper, we introduce Py-Feat, an open-source Python toolbox that provides support for detecting, preprocessing, analyzing, and visualizing facial expression data. Py-Feat makes it easy for domain experts to disseminate and benchmark computer vision models and also for end users to quickly process, analyze, and visualize face expression data. We hope this platform will facilitate increased use of facial expression data in human behavior research.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-023-00191-4.

Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra-facial information

Deep learning techniques can use public data such as facial photographs to predict sensitive personal information, but little is known about what information contributes to the predictive success of these techniques. This lack of knowledge limits both the public's ability to protect against revealing unintended information as well as the scientific utility of deep learning results. We combine convolutional neural networks, heat maps, facial expression coding, and classification of identifiable features such as masculinity and attractiveness in our study of political ideology in 3323 Danes. Predictive accuracy from the neural network was 61% in each gender. Model-predicted ideology correlated with aspects of both facial expressions (happiness vs neutrality) and morphology (specifically, attractiveness in females). Heat maps highlighted the informativeness of areas both on and off the face, pointing to methodological refinements and the need for future research to better understand the significance of certain facial areas.

Face Recognition Method under Adaptive Image Matching and Dictionary Learning Algorithm

In this research, a robust face recognition method based on adaptive image matching and a dictionary learning algorithm was proposed. A Fisher discriminant constraint was introduced into the dictionary learning algorithm program so that the dictionary had certain category discrimination ability. The purpose was to use this technology to reduce the influence of pollution, absence, and other factors on face recognition and improve the recognition rate. The optimization method was used to solve the loop iteration to obtain the expected specific dictionary, and the selected specific dictionary was used as the representation dictionary in adaptive sparse representation. In addition, if a specific dictionary was placed in a seed space of the original training data, the mapping matrix can be used to represent the mapping relationship between the specific dictionary and the original training sample, and the test sample could be corrected according to the mapping matrix to remove the contamination in the test sample. Moreover, the feature face method and dimension reduction method were used to process the specific dictionary and the corrected test sample, and the dimensions were reduced to 25, 50, 75, 100, 125, and 150, respectively. In this research, the recognition rate of the algorithm in 50 dimensions was lower than that of the discriminatory low-rank representation method (DLRR), and the recognition rate in other dimensions was the highest. The adaptive image matching classifier was used for classification and recognition. The experimental results showed that the proposed algorithm had a good recognition rate and good robustness against noise, pollution, and occlusion. Health condition prediction based on face recognition technology has the advantages of being noninvasive and convenient operation.

Registered report: Social face evaluation: ethnicity-specific differences in the judgement of trustworthiness of faces and facial parts

Social face evaluation is a common and consequential element of everyday life based on the judgement of trustworthiness. However, the particular facial regions that guide such trustworthiness judgements are largely unknown. It is also unclear whether different facial regions are consistently utilized to guide judgments for different ethnic groups, and whether previous exposure to specific ethnicities in one's social environment has an influence on trustworthiness judgements made from faces or facial regions. This registered report addressed these questions through a global online survey study that recruited Asian, Black, Latino, and White raters (N = 4580). Raters were shown full faces and specific parts of the face for an ethnically diverse, sex-balanced set of 32 targets and rated targets' trustworthiness. Multilevel modelling showed that in forming trustworthiness judgements, raters relied most strongly on the eyes (with no substantial information loss vis-à-vis full faces). Corroborating ingroup-outgroup effects, raters rated faces and facial parts of targets with whom they shared their ethnicity, sex, or eye color as significantly more trustworthy. Exposure to ethnic groups in raters' social environment predicted trustworthiness ratings of other ethnic groups in nuanced ways. That is, raters from the ambient ethnic majority provided slightly higher trustworthiness ratings for stimuli of their own ethnicity compared to minority ethnicities. In contrast, raters from an ambient ethnic minority (e.g., immigrants) provided substantially lower trustworthiness ratings for stimuli of the ethnic majority. Taken together, the current study provides a new window into the psychological processes underlying social face evaluation and its cultural generalizability. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on 7 January 2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.6084/m9.figshare.18319244 .

Algorithmic Political Bias in Artificial Intelligence Systems

Some artificial intelligence (AI) systems can display algorithmic bias, i.e. they may produce outputs that unfairly discriminate against people based on their social identity. Much research on this topic focuses on algorithmic bias that disadvantages people based on their gender or racial identity. The related ethical problems are significant and well known. Algorithmic bias against other aspects of people’s social identity, for instance, their political orientation, remains largely unexplored. This paper argues that algorithmic bias against people’s political orientation can arise in some of the same ways in which algorithmic gender and racial biases emerge. However, it differs importantly from them because there are (in a democratic society) strong social norms against gender and racial biases. This does not hold to the same extent for political biases. Political biases can thus more powerfully influence people, which increases the chances that these biases become embedded in algorithms and makes algorithmic political biases harder to detect and eradicate than gender and racial biases even though they all can produce similar harm. Since some algorithms can now also easily identify people’s political orientations against their will, these problems are exacerbated. Algorithmic political bias thus raises substantial and distinctive risks that the AI community should be aware of and examine.

Supporting Artificial Social Intelligence With Theory of Mind

In this paper, we discuss the development of artificial theory of mind as foundational to an agent's ability to collaborate with human team members. Agents imbued with artificial social intelligence will require various capabilities to gather the social data needed to inform an artificial theory of mind of their human counterparts. We draw from social signals theorizing and discuss a framework to guide consideration of core features of artificial social intelligence. We discuss how human social intelligence, and the development of theory of mind, can contribute to the development of artificial social intelligence by forming a foundation on which to help agents model, interpret and predict the behaviors and mental states of humans to support human-agent interaction. Artificial social intelligence will need the processing capabilities to perceive, interpret, and generate combinations of social cues to operate within a human-agent team. Artificial Theory of Mind affords a structure by which a socially intelligent agent could be imbued with the ability to model their human counterparts and engage in effective human-agent interaction. Further, modeling Artificial Theory of Mind can be used by an ASI to support transparent communication with humans, improving trust in agents, so that they may better predict future system behavior based on their understanding of and support trust in artificial socially intelligent agents.

Ethical problems in the use of algorithms in data management and in a free market economy

The problem that I present in this paper concerns the issue of ethical evaluation of algorithms, especially those used in social media and which create profiles of users of these media and new technologies that have recently emerged and are intended to change the functioning of technologies used in data management. Systems such as Overton, SambaNova or Snorkel were created to help engineers create data management models, but they are based on different assumptions than the previous approach in machine learning and deep learning. There is a need to analyze both deep learning algorithms and new technologies in database management in terms of their actions towards a person who leaves their digital footprints, on which these technologies work. Then, the possibilities of applying the existing deep learning technology and new Big Data systems in the economy will be shown. The opportunities offered by the systems mentioned above seem to be promising for many companies and—if implemented on a larger scale—they will affect the functioning of the free market.

Machine Learning Algorithms to Detect Sex in Myocardial Perfusion Imaging

Myocardial perfusion imaging (MPI) is an essential tool used to diagnose and manage patients with suspected or known coronary artery disease. Additionally, the General Data Protection Regulation (GDPR) represents a milestone about individuals' data security concerns. On the other hand, Machine Learning (ML) has had several applications in the most diverse knowledge areas. It is conceived as a technology with huge potential to revolutionize health care. In this context, we developed ML models to evaluate their ability to distinguish an individual's sex from MPI assessment. We used 260 polar maps (140 men/120 women) to train ML algorithms from a database of patients referred to a university hospital for clinically indicated MPI from January 2016 to December 2018. We tested 07 different ML models, namely, Classification and Regression Tree (CART), Naive Bayes (NB), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Adaptive Boosting (AB), Random Forests (RF) and, Gradient Boosting (GB). We used a cross-validation strategy. Our work demonstrated that ML algorithms could perform well in assessing the sex of patients undergoing myocardial scintigraphy exams. All the models had accuracy greater than 82%. However, only SVM achieved 90%. KNN, RF, AB, GB had, respectively, 88, 86, 85, 83%. Accuracy standard deviation was lower in KNN, AB, and RF (0.06). SVM and RF had had the best area under the receiver operating characteristic curve (0.93), followed by GB (0.92), KNN (0.91), AB, and NB (0.9). SVM and AB achieved the best precision. Our results bring some challenges regarding the autonomy of patients who wish to keep sex information confidential and certainly add greater complexity to the debate about what data should be considered sensitive to the light of the GDPR.

Can Facebook likes predict the purchase probability of electricity storage systems?

This study among owners of photovoltaic systems investigates whether users' Big Five personality traits derived from their Facebook likes contribute to whether or not they adopt an electricity storage. It is based on the finding that the digital footprint, especially the Facebook likes, can in part predict the personality of users better than friends and family. The survey was conducted among 159 Facebook users in Germany who owned a photovoltaic system. For comparison, a control sample with data from the German Socio-Economic Panel with 425 photovoltaic owners among 7286 individuals was used. The results show that, for extraversion, agreeableness, and neuroticism, the mean scores could be sufficiently predicted. However, a positive correlation could only be detected for extraversion. The comparison of the user groups could not provide satisfying results. None of the Big Five personality traits could be used to distinguish the two user groups from each other. Although the results did not support the hypotheses, this study offers insights into the possibilities of combining data mining, personality psychology, and consumer research.