1
|
Yovel G, Bash E, Bate S. Humans' extreme face recognition abilities challenge the well-established familiarity effect. Cognition 2024; 251:105904. [PMID: 39106626 DOI: 10.1016/j.cognition.2024.105904] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Revised: 06/15/2024] [Accepted: 07/22/2024] [Indexed: 08/09/2024]
Abstract
Classification performance is better for learned than unlearned stimuli. This was also reported for faces, where identity matching of unfamiliar faces is worse than for familiar faces. This familiarity advantage led to the conclusion that variability across appearances of the same identity is partly idiosyncratic and cannot be generalized from familiar to unfamiliar identities. Recent advances in machine vision challenge this claim by showing that the performance for untrained (unfamiliar) identities reached the level of trained identities as the number of identities that the algorithm is trained with increases. We therefore asked whether humans who reportedly can identify a vast number of identities, such as super recognizers, may close the gap between familiar and unfamiliar face classification. Consistent with this prediction, super recognizers classified unfamiliar faces just as well as typical participants who are familiar with the same faces, on a task that generates a sizable familiarity effect in controls. Additionally, prosopagnosics' performance for familiar faces was as bad as that of typical participants who were unfamiliar with the same faces, indicating that they struggle to learn even identity-specific information. Overall, these findings demonstrate that by studying the extreme ends of a system's ability we can gain novel insights into its actual capabilities.
Collapse
Affiliation(s)
- Gailt Yovel
- School of Psychological Sciences Tel Aviv University, Tel Aviv, Israel; Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel.
| | - Eden Bash
- School of Psychological Sciences Tel Aviv University, Tel Aviv, Israel
| | - Sarah Bate
- Department of Psychology, Bournemouth University, UK
| |
Collapse
|
2
|
Prunty JE, Jenkins R, Qarooni R, Bindemann M. A cognitive template for human face detection. Cognition 2024; 249:105792. [PMID: 38763070 DOI: 10.1016/j.cognition.2024.105792] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Revised: 02/05/2024] [Accepted: 04/08/2024] [Indexed: 05/21/2024]
Abstract
Faces are highly informative social stimuli, yet before any information can be accessed, the face must first be detected in the visual field. A detection template that serves this purpose must be able to accommodate the wide variety of face images we encounter, but how this generality could be achieved remains unknown. In this study, we investigate whether statistical averages of previously encountered faces can form the basis of a general face detection template. We provide converging evidence from a range of methods-human similarity judgements and PCA-based image analysis of face averages (Experiment 1-3), human detection behaviour for faces embedded in complex scenes (Experiment 4 and 5), and simulations with a template-matching algorithm (Experiment 6 and 7)-to examine the formation, stability and robustness of statistical image averages as cognitive templates for human face detection. We integrate these findings with existing knowledge of face identification, ensemble coding, and the development of face perception.
Collapse
Affiliation(s)
- Jonathan E Prunty
- Leverhulme Centre for the Future of Intelligence, University of Cambridge, Cambridge, UK.
| | - Rob Jenkins
- Department of Psychology, University of York, York, UK
| | - Rana Qarooni
- Department of Psychology, University of York, York, UK
| | | |
Collapse
|
3
|
Rozado D. The political preferences of LLMs. PLoS One 2024; 19:e0306621. [PMID: 39083484 PMCID: PMC11290627 DOI: 10.1371/journal.pone.0306621] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2024] [Accepted: 06/18/2024] [Indexed: 08/02/2024] Open
Abstract
I report here a comprehensive analysis about the political preferences embedded in Large Language Models (LLMs). Namely, I administer 11 political orientation tests, designed to identify the political preferences of the test taker, to 24 state-of-the-art conversational LLMs, both closed and open source. When probed with questions/statements with political connotations, most conversational LLMs tend to generate responses that are diagnosed by most political test instruments as manifesting preferences for left-of-center viewpoints. This does not appear to be the case for five additional base (i.e. foundation) models upon which LLMs optimized for conversation with humans are built. However, the weak performance of the base models at coherently answering the tests' questions makes this subset of results inconclusive. Finally, I demonstrate that LLMs can be steered towards specific locations in the political spectrum through Supervised Fine-Tuning (SFT) with only modest amounts of politically aligned data, suggesting SFT's potential to embed political orientation in LLMs. With LLMs beginning to partially displace traditional information sources like search engines and Wikipedia, the societal implications of political biases embedded in LLMs are substantial.
Collapse
|
4
|
Yang M, El-Attar AA, Chaspari T. Deconstructing demographic bias in speech-based machine learning models for digital health. Front Digit Health 2024; 6:1351637. [PMID: 39119589 PMCID: PMC11306200 DOI: 10.3389/fdgth.2024.1351637] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Accepted: 07/15/2024] [Indexed: 08/10/2024] Open
Abstract
Introduction Machine learning (ML) algorithms have been heralded as promising solutions to the realization of assistive systems in digital healthcare, due to their ability to detect fine-grain patterns that are not easily perceived by humans. Yet, ML algorithms have also been critiqued for treating individuals differently based on their demography, thus propagating existing disparities. This paper explores gender and race bias in speech-based ML algorithms that detect behavioral and mental health outcomes. Methods This paper examines potential sources of bias in the data used to train the ML, encompassing acoustic features extracted from speech signals and associated labels, as well as in the ML decisions. The paper further examines approaches to reduce existing bias via using the features that are the least informative of one's demographic information as the ML input, and transforming the feature space in an adversarial manner to diminish the evidence of the demographic information while retaining information about the focal behavioral and mental health state. Results Results are presented in two domains, the first pertaining to gender and race bias when estimating levels of anxiety, and the second pertaining to gender bias in detecting depression. Findings indicate the presence of statistically significant differences in both acoustic features and labels among demographic groups, as well as differential ML performance among groups. The statistically significant differences present in the label space are partially preserved in the ML decisions. Although variations in ML performance across demographic groups were noted, results are mixed regarding the models' ability to accurately estimate healthcare outcomes for the sensitive groups. Discussion These findings underscore the necessity for careful and thoughtful design in developing ML models that are capable of maintaining crucial aspects of the data and perform effectively across all populations in digital healthcare applications.
Collapse
Affiliation(s)
- Michael Yang
- Computer Science & Engineering, Texas A&M University, College Station, TX, United States
| | - Abd-Allah El-Attar
- Computer Science & Engineering, Texas A&M University Qatar, Al Rayyan, Qatar
| | - Theodora Chaspari
- Institute of Cognitive Science & Computer Science, University of Colorado Boulder, Boulder, CO, United States
| |
Collapse
|
5
|
Alfano M, Abedin E, Reimann R, Ferreira M, Cheong M. Now you see me, now you don't: an exploration of religious exnomination in DALL-E. ETHICS AND INFORMATION TECHNOLOGY 2024; 26:27. [PMID: 38617999 PMCID: PMC11009767 DOI: 10.1007/s10676-024-09760-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 03/09/2024] [Indexed: 04/16/2024]
Abstract
Artificial intelligence (AI) systems are increasingly being used not only to classify and analyze but also to generate images and text. As recent work on the content produced by text and image Generative AIs has shown (e.g., Cheong et al., 2024, Acerbi & Stubbersfield, 2023), there is a risk that harms of representation and bias, already documented in prior AI and natural language processing (NLP) algorithms may also be present in generative models. These harms relate to protected categories such as gender, race, age, and religion. There are several kinds of harms of representation to consider in this context, including stereotyping, lack of recognition, denigration, under-representation, and many others (Crawford in Soundings 41:45-55, 2009; in: Barocas et al., SIGCIS Conference, 2017). Whereas the bulk of researchers' attention thus far has been given to stereotyping and denigration, in this study we examine 'exnomination', as conceived by Roland Barthes (1972), of religious groups. Our case study is DALL-E, a tool that generates images from natural language prompts. Using DALL-E mini, we generate images from generic prompts such as "religious person." We then examine whether the generated images are recognizably members of a nominated group. Thus, we assess whether the generated images normalize some religions while neglecting others. We hypothesize that Christianity will be recognizably represented more frequently than other religious groups. Our results partially support this hypothesis but introduce further complexities, which we then explore.
Collapse
Affiliation(s)
- Mark Alfano
- Department of Philosophy, Macquarie University, Sydney, NSW Australia
| | - Ehsan Abedin
- School of Computing and Information Systems, University of Melbourne, Melbourne, VIC Australia
- College of Business, Government, and Law, Flinders University, Adelaide, SA Australia
| | - Ritsaart Reimann
- Department of Philosophy, Macquarie University, Sydney, NSW Australia
| | - Marinus Ferreira
- Department of Philosophy, Macquarie University, Sydney, NSW Australia
| | - Marc Cheong
- School of Computing and Information Systems, University of Melbourne, Melbourne, VIC Australia
| |
Collapse
|
6
|
Mallick S, Jeckeln G, Parde CJ, Castillo CD, O'Toole AJ. The Influence of the Other-Race Effect on Susceptibility to Face Morphing Attacks. ACM TRANSACTIONS ON APPLIED PERCEPTION 2024; 21:2. [PMID: 39131565 PMCID: PMC11315460 DOI: 10.1145/3618113] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Accepted: 08/24/2023] [Indexed: 08/13/2024]
Abstract
Facial morphs created between two identities resemble both of the faces used to create the morph. Consequently, humans and machines are prone to mistake morphs made from two identities for either of the faces used to create the morph. This vulnerability has been exploited in "morph attacks" in security scenarios. Here, we asked whether the "other-race effect" (ORE)-the human advantage for identifying own- vs. other-race faces-exacerbates morph attack susceptibility for humans. We also asked whether face-identification performance in a deep convolutional neural network (DCNN) is affected by the race of morphed faces. Caucasian (CA) and East-Asian (EA) participants performed a face-identity matching task on pairs of CA and EA face images in two conditions. In the morph condition, different-identity pairs consisted of an image of identity "A" and a 50/50 morph between images of identity "A" and "B". In the baseline condition, morphs of different identities never appeared. As expected, morphs were identified mistakenly more often than original face images. Of primary interest, morph identification was substantially worse for cross-race faces than for own-race faces. Similar to humans, the DCNN performed more accurately for original face images than for morphed image pairs. Notably, the deep network proved substantially more accurate than humans in both cases. The results point to the possibility that DCNNs might be useful for improving face identification accuracy when morphed faces are presented. They also indicate the significance of the race of a face in morph attack susceptibility in applied settings.
Collapse
Affiliation(s)
- Snipta Mallick
- School of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX
| | - Géraldine Jeckeln
- School of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX
| | - Connor J Parde
- School of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX
| | - Carlos D Castillo
- Whiting School of Engineering, Johns Hopkins University, College Park, MD
| | - Alice J O'Toole
- School of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, TX
| |
Collapse
|
7
|
Hiba S, Keller Y. Hierarchical Attention-Based Age Estimation and Bias Analysis. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2023; 45:14682-14692. [PMID: 37751349 DOI: 10.1109/tpami.2023.3319472] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/28/2023]
Abstract
In this work, we present a Deep Learning approach to estimate age from facial images. First, we introduce a novel attention-based approach to image augmentation-aggregation, which allows multiple image augmentations to be adaptively aggregated using a Transformer-Encoder. A hierarchical probabilistic regression model is then proposed that combines discrete probabilistic age estimates with an ensemble of regressors. Each regressor is adapted and trained to refine the probability estimate over a given age range. We show that our age estimation scheme outperforms current schemes and provides a new state-of-the-art age estimation accuracy when applied to the MORPH II and CACD datasets. We also present an analysis of the biases in the results of the state-of-the-art age estimates.
Collapse
|
8
|
Wang A, Sliwinska MW, Watson DM, Smith S, Andrews TJ. Distinct patterns of neural response to faces from different races in humans and deep networks. Soc Cogn Affect Neurosci 2023; 18:nsad059. [PMID: 37837305 PMCID: PMC10634630 DOI: 10.1093/scan/nsad059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 07/27/2023] [Accepted: 10/06/2023] [Indexed: 10/15/2023] Open
Abstract
Social categories such as the race or ethnicity of an individual are typically conveyed by the visual appearance of the face. The aim of this study was to explore how these differences in facial appearance are represented in human and artificial neural networks. First, we compared the similarity of faces from different races using a neural network trained to discriminate identity. We found that the differences between races were most evident in the fully connected layers of the network. Although these layers were also able to predict behavioural judgements of face identity from human participants, performance was biased toward White faces. Next, we measured the neural response in face-selective regions of the human brain to faces from different races in Asian and White participants. We found distinct patterns of response to faces from different races in face-selective regions. We also found that the spatial pattern of response was more consistent across participants for own-race compared to other-race faces. Together, these findings show that faces from different races elicit different patterns of response in human and artificial neural networks. These differences may underlie the ability to make categorical judgements and explain the behavioural advantage for the recognition of own-race faces.
Collapse
Affiliation(s)
- Ao Wang
- Department of Psychology, University of York, York YO10 5DD, UK
- Department of Psychology, University of Southampton, Southampton SO17 1BJ, UK
| | - Magdalena W Sliwinska
- Department of Psychology, University of York, York YO10 5DD, UK
- School of Psychology, Liverpool John Moores University, Liverpool L2 2QP, UK
| | - David M Watson
- Department of Psychology, University of York, York YO10 5DD, UK
| | - Sam Smith
- Department of Psychology, University of York, York YO10 5DD, UK
| | | |
Collapse
|
9
|
Nephew BC, Polcari JJ, Korkin D. Ideography insight from facial recognition and neuroimaging. Behav Brain Sci 2023; 46:e249. [PMID: 37779279 DOI: 10.1017/s0140525x23000602] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/03/2023]
Abstract
One novel example and/or perspective in support of "Why the learning account fails" is the impressive ability of humans to recognize and memorize facial features and accurately and reliably connect those to related identities. Furthermore, neuroimaging analysis presents an example in support of the crucial role of standardization in the lack of adoption of ideography.
Collapse
Affiliation(s)
- Benjamin C Nephew
- Department of Biology and Biotechnology, Worcester Polytechnic Institute, Worcester, MA, USA
| | - Justin J Polcari
- Department of Biology and Biotechnology, Worcester Polytechnic Institute, Worcester, MA, USA
| | - Dmitry Korkin
- Department of Computer Science, Worcester Polytechnic Institute, Worcester, MA, USA
| |
Collapse
|
10
|
Dobs K, Yuan J, Martinez J, Kanwisher N. Behavioral signatures of face perception emerge in deep neural networks optimized for face recognition. Proc Natl Acad Sci U S A 2023; 120:e2220642120. [PMID: 37523537 PMCID: PMC10410721 DOI: 10.1073/pnas.2220642120] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 06/08/2023] [Indexed: 08/02/2023] Open
Abstract
Human face recognition is highly accurate and exhibits a number of distinctive and well-documented behavioral "signatures" such as the use of a characteristic representational space, the disproportionate performance cost when stimuli are presented upside down, and the drop in accuracy for faces from races the participant is less familiar with. These and other phenomena have long been taken as evidence that face recognition is "special". But why does human face perception exhibit these properties in the first place? Here, we use deep convolutional neural networks (CNNs) to test the hypothesis that all of these signatures of human face perception result from optimization for the task of face recognition. Indeed, as predicted by this hypothesis, these phenomena are all found in CNNs trained on face recognition, but not in CNNs trained on object recognition, even when additionally trained to detect faces while matching the amount of face experience. To test whether these signatures are in principle specific to faces, we optimized a CNN on car discrimination and tested it on upright and inverted car images. As we found for face perception, the car-trained network showed a drop in performance for inverted vs. upright cars. Similarly, CNNs trained on inverted faces produced an inverted face inversion effect. These findings show that the behavioral signatures of human face perception reflect and are well explained as the result of optimization for the task of face recognition, and that the nature of the computations underlying this task may not be so special after all.
Collapse
Affiliation(s)
- Katharina Dobs
- Department of Psychology, Justus Liebig University Giessen, Giessen35394, Germany
- Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University Giessen, Marburg35302, Germany
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA02139
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA02139
| | - Joanne Yuan
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA02139
| | - Julio Martinez
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA02139
- Department of Psychology, Stanford University, Stanford, CA94305
| | - Nancy Kanwisher
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA02139
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA02139
| |
Collapse
|
11
|
Robinson JP, Qin C, Henon Y, Timoner S, Fu Y. Balancing Biases and Preserving Privacy on Balanced Faces in the Wild. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2023; 32:4365-4377. [PMID: 37467097 DOI: 10.1109/tip.2023.3282837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/21/2023]
Abstract
There are demographic biases present in current facial recognition (FR) models. To measure these biases across different ethnic and gender subgroups, we introduce our Balanced Faces in the Wild (BFW) dataset. This dataset allows for the characterization of FR performance per subgroup. We found that relying on a single score threshold to differentiate between genuine and imposters sample pairs leads to suboptimal results. Additionally, performance within subgroups often varies significantly from the global average. Therefore, specific error rates only hold for populations that match the validation data. To mitigate imbalanced performances, we propose a novel domain adaptation learning scheme that uses facial features extracted from state-of-the-art neural networks. This scheme boosts the average performance and preserves identity information while removing demographic knowledge. Removing demographic knowledge prevents potential biases from affecting decision-making and protects privacy by eliminating demographic information. We explore the proposed method and demonstrate that subgroup classifiers can no longer learn from features projected using our domain adaptation scheme. For access to the source code and data, please visit https://github.com/visionjo/facerec-bias-bfw.
Collapse
|
12
|
Muthua AG, Theart RP, Booysen MJ. Using infrared to improve face recognition of individuals with highly pigmented skin. iScience 2023; 26:107039. [PMID: 37416460 PMCID: PMC10320199 DOI: 10.1016/j.isci.2023.107039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 03/22/2023] [Accepted: 05/31/2023] [Indexed: 07/08/2023] Open
Abstract
Face recognition is widely used for security and access control. Its performance is limited when working with highly pigmented skin tones due to training bias caused by the under-representation of darker-skinned individuals in existing datasets and the fact that darker skin absorbs more light and therefore reflects less discernible detail in the visible spectrum. To improve performance, this work incorporated the infrared (IR) spectrum, which is perceived by electronic sensors. We augmented existing datasets with images of highly pigmented individuals captured using the visible, IR, and full spectra and fine-tuned existing face recognition systems to compare the performance of these three. We found a marked improvement in accuracy and AUC values of the receiver operating characteristic (ROC) curves when including the IR spectrum, increasing performance from 97.5% to 99.0% for highly pigmented faces. Different facial orientations and narrow cropping also improved performance, and the nose region was the most important feature for recognition.
Collapse
Affiliation(s)
- Alex G. Muthua
- Department of Electrical and Electronic Engineering, Stellenbosch Universiteit, Stellenbosch, South Africa
| | - Rensu P. Theart
- Department of Electrical and Electronic Engineering, Stellenbosch Universiteit, Stellenbosch, South Africa
| | - Marthinus J. Booysen
- Department of Electrical and Electronic Engineering, Stellenbosch Universiteit, Stellenbosch, South Africa
- Department of Industrial Engineering, Stellenbosch University, Stellenbosch, South Africa
| |
Collapse
|
13
|
Parde CJ, Strehle VE, Banerjee V, Hu Y, Cavazos JG, Castillo CD, O'Toole AJ. Twin Identification over Viewpoint Change: A Deep Convolutional Neural Network Surpasses Humans. ACM TRANSACTIONS ON APPLIED PERCEPTION 2023; 20:10. [PMID: 39131580 PMCID: PMC11315461 DOI: 10.1145/3609224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 04/30/2023] [Indexed: 08/13/2024]
Abstract
Deep convolutional neural networks (DCNNs) have achieved human-level accuracy in face identification (Phillips et al., 2018), though it is unclear how accurately they discriminate highly-similar faces. Here, humans and a DCNN performed a challenging face-identity matching task that included identical twins. Participants (N = 87) viewed pairs of face images of three types: same-identity, general imposters (different identities from similar demographic groups), and twin imposters (identical twin siblings). The task was to determine whether the pairs showed the same person or different people. Identity comparisons were tested in three viewpoint-disparity conditions: frontal to frontal, frontal to 45° profile, and frontal to 90°profile. Accuracy for discriminating matched-identity pairs from twin-imposter pairs and general-imposter pairs was assessed in each viewpoint-disparity condition. Humans were more accurate for general-imposter pairs than twin-imposter pairs, and accuracy declined with increased viewpoint disparity between the images in a pair. A DCNN trained for face identification (Ranjan et al., 2018) was tested on the same image pairs presented to humans. Machine performance mirrored the pattern of human accuracy, but with performance at or above all humans in all but one condition. Human and machine similarity scores were compared across all image-pair types. This item-level analysis showed that human and machine similarity ratings correlated significantly in six of nine image-pair types [range r = 0.38 to r = 0.63], suggesting general accord between the perception of face similarity by humans and the DCNN. These findings also contribute to our understanding of DCNN performance for discriminating high-resemblance faces, demonstrate that the DCNN performs at a level at or above humans, and suggest a degree of parity between the features used by humans and the DCNN.
Collapse
Affiliation(s)
- Connor J Parde
- School of Behavioral and Brain Sciences, The University of Texas at Dallas, USA
| | - Virginia E Strehle
- School of Behavioral and Brain Sciences, The University of Texas at Dallas, USA
| | - Vivekjyoti Banerjee
- University of Maryland Institute of Advanced Computer Studies, University of Maryland, USA
| | - Ying Hu
- School of Behavioral and Brain Sciences, The University of Texas at Dallas, USA
| | | | | | - Alice J O'Toole
- School of Behavioral and Brain Sciences, The University of Texas at Dallas, USA
| |
Collapse
|
14
|
Baker KA, Stabile VJ, Mondloch CJ. Stable individual differences in unfamiliar face identification: Evidence from simultaneous and sequential matching tasks. Cognition 2023; 232:105333. [PMID: 36508992 DOI: 10.1016/j.cognition.2022.105333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Revised: 11/14/2022] [Accepted: 11/19/2022] [Indexed: 12/14/2022]
Abstract
Matching identity in images of unfamiliar faces is difficult: Images of the same person can look different and images of different people can look similar. Recent studies have capitalized on individual differences in the ability to distinguish match (same ID) vs. mismatch (different IDs) face pairs to inform models of face recognition. We addressed two significant gaps in the literature by examining the stability of individual differences in both sensitivity to identity and response bias. In Study 1, 210 participants completed a battery of four tasks in each of two sessions separated by one week. Tasks varied in protocol (same/different, lineup, sorting) and stimulus characteristics (low vs. high within-person variability in appearance). In Study 2, 148 participants completed a battery of three tasks in a single session. Stimuli were presented simultaneously on some trials and sequentially on others, introducing short-term memory demands. Principal components analysis revealed two components that were stable across time and tasks: sensitivity to identity and bias. Analyses of response times suggest that individual differences in bias reflect decision-making processes. We discuss the implications of our findings in applied settings and for models of face recognition.
Collapse
|
15
|
Eyiokur FI, Kantarcı A, Erakın ME, Damer N, Ofli F, Imran M, Križaj J, Salah AA, Waibel A, Štruc V, Ekenel HK. A survey on computer vision based human analysis in the COVID-19 era. IMAGE AND VISION COMPUTING 2023; 130:104610. [PMID: 36540857 PMCID: PMC9755265 DOI: 10.1016/j.imavis.2022.104610] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 12/11/2022] [Indexed: 06/17/2023]
Abstract
The emergence of COVID-19 has had a global and profound impact, not only on society as a whole, but also on the lives of individuals. Various prevention measures were introduced around the world to limit the transmission of the disease, including face masks, mandates for social distancing and regular disinfection in public spaces, and the use of screening applications. These developments also triggered the need for novel and improved computer vision techniques capable of ( i ) providing support to the prevention measures through an automated analysis of visual data, on the one hand, and ( ii ) facilitating normal operation of existing vision-based services, such as biometric authentication schemes, on the other. Especially important here, are computer vision techniques that focus on the analysis of people and faces in visual data and have been affected the most by the partial occlusions introduced by the mandates for facial masks. Such computer vision based human analysis techniques include face and face-mask detection approaches, face recognition techniques, crowd counting solutions, age and expression estimation procedures, models for detecting face-hand interactions and many others, and have seen considerable attention over recent years. The goal of this survey is to provide an introduction to the problems induced by COVID-19 into such research and to present a comprehensive review of the work done in the computer vision based human analysis field. Particular attention is paid to the impact of facial masks on the performance of various methods and recent solutions to mitigate this problem. Additionally, a detailed review of existing datasets useful for the development and evaluation of methods for COVID-19 related applications is also provided. Finally, to help advance the field further, a discussion on the main open challenges and future research direction is given at the end of the survey. This work is intended to have a broad appeal and be useful not only for computer vision researchers but also the general public.
Collapse
Affiliation(s)
- Fevziye Irem Eyiokur
- Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Karlsruhe, Germany
| | - Alperen Kantarcı
- Department of Computer Engineering, Istanbul Technical University, Istanbul, Turkey
| | - Mustafa Ekrem Erakın
- Department of Computer Engineering, Istanbul Technical University, Istanbul, Turkey
| | - Naser Damer
- Fraunhofer Institute for Computer Graphics Research IGD, Darmstadt, Germany
- Department of Computer Science, TU Darmstadt, Darmstadt, Germany
| | - Ferda Ofli
- Qatar Computing Research Institute, HBKU, Doha, Qatar
| | | | - Janez Križaj
- Faculty of Electrical Engineering, University of Ljubljana, Tržaška cesta 25, 1000 Ljubljana, Slovenia
| | - Albert Ali Salah
- Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands
- Department of Computer Engineering, Bogˇaziçi University, Istanbul, Turkey
| | - Alexander Waibel
- Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Karlsruhe, Germany
- Carnegie Mellon University, Pittsburgh, United States
| | - Vitomir Štruc
- Faculty of Electrical Engineering, University of Ljubljana, Tržaška cesta 25, 1000 Ljubljana, Slovenia
| | - Hazım Kemal Ekenel
- Department of Computer Engineering, Istanbul Technical University, Istanbul, Turkey
| |
Collapse
|
16
|
Perez O, Garza T, Hindera O, Beltran A, Musaad SM, Dibbs T, Singh A, Chug S, Sisson A, Kumar Vadathya A, Baranowski T, O'Connor TM. Validated assessment tools for screen media use: A systematic review. PLoS One 2023; 18:e0283714. [PMID: 37053175 PMCID: PMC10101444 DOI: 10.1371/journal.pone.0283714] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Accepted: 03/15/2023] [Indexed: 04/14/2023] Open
Abstract
OBJECTIVE Accurate measurement of adult and child screen media use are needed to robustly assess its impact on health outcomes. Our objective was to systematically review screen media use measurement tools that have been validated against an objective "gold standard" tool. METHODS The search strategy was initially conducted in Medline Ovid and translated to Embase, Web of Science, PsychInfo and Cochrane. A modified natural language search was conducted in Google Scholar and IEEE. The initial search was conducted in March 2021, and an updated search was conducted in June 2022. Additional studies were included from the references. Studies had to describe the validation of a tool to measure screen media use on participants of any age against a 'gold standard' or comparable objective measure. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) was used to assess the criterion validity. Four authors reviewed the titles in two rounds and extracted data. RESULTS Twenty-nine articles were included in the review. Studies measured TV, computer, mobile device and social media site screen media use through: self or parent report, direct or video observation, computer and mobile device use tracking programs, and through other novel devices such as wearable devices and cameras. Correlations of self or parent report of screen media with the gold standard were lower than correlations of technology-based measures, and video observation with the gold standard. The COSMIN criterion validity ratings ranged from poor to excellent; most of the studies received a global score of fair or poor. CONCLUSIONS Technology based validated tools that more directly measure screen use are emerging that have been validated against a gold standard for measuring screen use. However, practical, objective measures of diverse types of screen media use that have been tested on diverse populations are needed to better understand the impact of screen media use on the development and physical and mental health of children and adults.
Collapse
Affiliation(s)
- Oriana Perez
- USDA/ARS Children's Nutrition Research Center, Baylor College of Medicine, Houston, Texas, United States of America
| | - Tatyana Garza
- USDA/ARS Children's Nutrition Research Center, Baylor College of Medicine, Houston, Texas, United States of America
| | - Olivia Hindera
- Baylor College of Medicine, Houston, Texas, United States of America
| | - Alicia Beltran
- USDA/ARS Children's Nutrition Research Center, Baylor College of Medicine, Houston, Texas, United States of America
| | - Salma M Musaad
- USDA/ARS Children's Nutrition Research Center, Baylor College of Medicine, Houston, Texas, United States of America
| | - Tracey Dibbs
- Rice University, Houston, Texas, United States of America
| | - Anu Singh
- Rice University, Houston, Texas, United States of America
| | - Shria Chug
- Rice University, Houston, Texas, United States of America
| | - Amy Sisson
- The Texas Medical Center Library, Houston, Texas, United States of America
| | - Anil Kumar Vadathya
- Rice University, Electrical & Computer Engineering, Houston, Texas, United States of America
| | - Tom Baranowski
- USDA/ARS Children's Nutrition Research Center, Baylor College of Medicine, Houston, Texas, United States of America
| | - Teresia M O'Connor
- USDA/ARS Children's Nutrition Research Center, Baylor College of Medicine, Houston, Texas, United States of America
| |
Collapse
|
17
|
Using Artificial Intelligence to Analyze Non-Human Drawings: A First Step with Orangutan Productions. Animals (Basel) 2022; 12:ani12202761. [PMID: 36290146 PMCID: PMC9597765 DOI: 10.3390/ani12202761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Revised: 10/06/2022] [Accepted: 10/11/2022] [Indexed: 11/07/2022] Open
Abstract
Simple Summary Understanding drawing features is a complex task, particularly concerning non-human primates, where the relevant features may not be the same as those for humans. Here, we propose a methodology for objectively analyzing drawings. To do so, we used deep learning, which allows for automated feature selection and extraction, to classify a female orangutan’s drawings according to the seasons they were produced. We found evidence of seasonal variation in her drawing behavior according to the extracted features, and our results support previous findings that features linked to colors can partly explain seasonal variation. Using grayscale images, we demonstrate that not only do colors contain relevant information but also the shape of the drawings. In addition, this study demonstrates that both the style and content of drawings partly explain seasonal variations. Abstract Drawings have been widely used as a window to the mind; as such, they can reveal some aspects of the cognitive and emotional worlds of other animals that can produce them. The study of non-human drawings, however, is limited by human perception, which can bias the methodology and interpretation of the results. Artificial intelligence can circumvent this issue by allowing automated, objective selection of features used to analyze drawings. In this study, we use artificial intelligence to investigate seasonal variations in drawings made by Molly, a female orangutan who produced more than 1299 drawings between 2006 and 2011 at the Tama Zoological Park in Japan. We train the VGG19 model to first classify the drawings according to the season in which they are produced. The results show that deep learning is able to identify subtle but significant seasonal variations in Molly’s drawings, with a classification accuracy of 41.6%. We use VGG19 to investigate the features that influence this seasonal variation. We analyze separate features, both simple and complex, related to color and patterning, and to drawing content and style. Content and style classification show maximum performance for moderately complex, highly complex, and holistic features, respectively. We also show that both color and patterning drive seasonal variation, with the latter being more important than the former. This study demonstrates how deep learning can be used to objectively analyze non-figurative drawings and calls for applications to non-primate species and scribbles made by human toddlers.
Collapse
|
18
|
Detecting racial inequalities in criminal justice: towards an equitable deep learning approach for generating and interpreting racial categories using mugshots. AI & SOCIETY 2022. [DOI: 10.1007/s00146-022-01440-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
19
|
Golder S, Stevens R, O'Connor K, James R, Gonzalez-Hernandez G. Methods to Establish Race or Ethnicity of Twitter Users: Scoping Review. J Med Internet Res 2022; 24:e35788. [PMID: 35486433 PMCID: PMC9107046 DOI: 10.2196/35788] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Revised: 03/08/2022] [Accepted: 03/23/2022] [Indexed: 11/13/2022] Open
Abstract
Background A growing amount of health research uses social media data. Those critical of social media research often cite that it may be unrepresentative of the population; however, the suitability of social media data in digital epidemiology is more nuanced. Identifying the demographics of social media users can help establish representativeness. Objective This study aims to identify the different approaches or combination of approaches to extract race or ethnicity from social media and report on the challenges of using these methods. Methods We present a scoping review to identify methods used to extract the race or ethnicity of Twitter users from Twitter data sets. We searched 17 electronic databases from the date of inception to May 15, 2021, and carried out reference checking and hand searching to identify relevant studies. Sifting of each record was performed independently by at least two researchers, with any disagreement discussed. Studies were required to extract the race or ethnicity of Twitter users using either manual or computational methods or a combination of both. Results Of the 1249 records sifted, we identified 67 (5.36%) that met our inclusion criteria. Most studies (51/67, 76%) have focused on US-based users and English language tweets (52/67, 78%). A range of data was used, including Twitter profile metadata, such as names, pictures, information from bios (including self-declarations), or location or content of the tweets. A range of methodologies was used, including manual inference, linkage to census data, commercial software, language or dialect recognition, or machine learning or natural language processing. However, not all studies have evaluated these methods. Those that evaluated these methods found accuracy to vary from 45% to 93% with significantly lower accuracy in identifying categories of people of color. The inference of race or ethnicity raises important ethical questions, which can be exacerbated by the data and methods used. The comparative accuracies of the different methods are also largely unknown. Conclusions There is no standard accepted approach or current guidelines for extracting or inferring the race or ethnicity of Twitter users. Social media researchers must carefully interpret race or ethnicity and not overpromise what can be achieved, as even manual screening is a subjective, imperfect method. Future research should establish the accuracy of methods to inform evidence-based best practice guidelines for social media researchers and be guided by concerns of equity and social justice.
Collapse
Affiliation(s)
- Su Golder
- Department of Health Sciences, University of York, York, United Kingdom
| | - Robin Stevens
- School of Communication and Journalism, University of Southern California, Los Angeles, CA, United States
| | - Karen O'Connor
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Richard James
- School of Nursing Liaison and Clinical Outreach Coordinator, University of Pennsylvania, Philadelphia, PA, United States
| | - Graciela Gonzalez-Hernandez
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| |
Collapse
|
20
|
Vadathya AK, Musaad S, Beltran A, Perez O, Meister L, Baranowski T, Hughes SO, Mendoza JA, Sabharwal A, Veeraraghavan A, O'Connor T. An Objective System for Quantitative Assessment of Television Viewing Among Children (Family Level Assessment of Screen Use in the Home-Television): System Development Study. JMIR Pediatr Parent 2022; 5:e33569. [PMID: 35323113 PMCID: PMC8990369 DOI: 10.2196/33569] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/13/2021] [Accepted: 01/11/2022] [Indexed: 12/26/2022] Open
Abstract
BACKGROUND Television viewing among children is associated with developmental and health outcomes, yet measurement techniques for television viewing are prone to errors, biases, or both. OBJECTIVE This study aims to develop a system to objectively and passively measure children's television viewing time. METHODS The Family Level Assessment of Screen Use in the Home-Television (FLASH-TV) system includes three sequential algorithms applied to video data collected in front of a television screen: face detection, face verification, and gaze estimation. A total of 21 families of diverse race and ethnicity were enrolled in 1 of 4 design studies to train the algorithms and provide proof of concept testing for the integrated FLASH-TV system. Video data were collected from each family in a laboratory mimicking a living room or in the child's home. Staff coded the video data for the target child as the gold standard. The accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were calculated for each algorithm, as compared with the gold standard. Prevalence and biased adjusted κ scores and an intraclass correlation using a generalized linear mixed model compared FLASH-TV's estimation of television viewing duration to the gold standard. RESULTS FLASH-TV demonstrated high sensitivity for detecting faces (95.5%-97.9%) and performed well on face verification when the child's gaze was on the television. Each of the metrics for estimating the child's gaze on the screen was moderate to good (range: 55.1% negative predictive value to 91.2% specificity). When combining the 3 sequential steps, FLASH-TV estimation of the child's screen viewing was overall good, with an intraclass correlation for an overall time watching television of 0.725 across conditions. CONCLUSIONS FLASH-TV offers a critical step forward in improving the assessment of children's television viewing.
Collapse
Affiliation(s)
- Anil Kumar Vadathya
- Department of Electrical & Computer Engineering, Rice University, Houston, TX, United States
| | - Salma Musaad
- Agricultural Research Service, US Department of Agriculture, Children's Nutrition Research Center, Baylor College of Medicine, Houston, TX, United States
| | - Alicia Beltran
- Agricultural Research Service, US Department of Agriculture, Children's Nutrition Research Center, Baylor College of Medicine, Houston, TX, United States
| | - Oriana Perez
- Agricultural Research Service, US Department of Agriculture, Children's Nutrition Research Center, Baylor College of Medicine, Houston, TX, United States
| | - Leo Meister
- Department of Electrical & Computer Engineering, Rice University, Houston, TX, United States
| | - Tom Baranowski
- Agricultural Research Service, US Department of Agriculture, Children's Nutrition Research Center, Baylor College of Medicine, Houston, TX, United States
| | - Sheryl O Hughes
- Agricultural Research Service, US Department of Agriculture, Children's Nutrition Research Center, Baylor College of Medicine, Houston, TX, United States
| | - Jason A Mendoza
- Public Health Sciences Division, Fred Hutchinson Cancer Research Center, Seattle, WA, United States.,General Pediatrics, Department of Pediatrics, University of Washington, Seattle, WA, United States
| | - Ashutosh Sabharwal
- Department of Electrical & Computer Engineering, Rice University, Houston, TX, United States
| | - Ashok Veeraraghavan
- Department of Electrical & Computer Engineering, Rice University, Houston, TX, United States
| | - Teresia O'Connor
- Agricultural Research Service, US Department of Agriculture, Children's Nutrition Research Center, Baylor College of Medicine, Houston, TX, United States
| |
Collapse
|
21
|
Ritchie KL, Cartledge C, Growns B, Yan A, Wang Y, Guo K, Kramer RSS, Edmond G, Martire KA, San Roque M, White D. Public attitudes towards the use of automatic facial recognition technology in criminal justice systems around the world. PLoS One 2021; 16:e0258241. [PMID: 34644306 PMCID: PMC8513835 DOI: 10.1371/journal.pone.0258241] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2021] [Accepted: 09/23/2021] [Indexed: 11/18/2022] Open
Abstract
Automatic facial recognition technology (AFR) is increasingly used in criminal justice systems around the world, yet to date there has not been an international survey of public attitudes toward its use. In Study 1, we ran focus groups in the UK, Australia and China (countries at different stages of adopting AFR) and in Study 2 we collected data from over 3,000 participants in the UK, Australia and the USA using a questionnaire investigating attitudes towards AFR use in criminal justice systems. Our results showed that although overall participants were aligned in their attitudes and reasoning behind them, there were some key differences across countries. People in the USA were more accepting of tracking citizens, more accepting of private companies’ use of AFR, and less trusting of the police using AFR than people in the UK and Australia. Our results showed that support for the use of AFR depends greatly on what the technology is used for and who it is used by. We recommend vendors and users do more to explain AFR use, including details around accuracy and data protection. We also recommend that governments should set legal boundaries around the use of AFR in investigative and criminal justice settings.
Collapse
Affiliation(s)
- Kay L. Ritchie
- School of Psychology, University of Lincoln, Lincoln, Lincolnshire, United Kingdom
- * E-mail:
| | - Charlotte Cartledge
- School of Psychology, University of Lincoln, Lincoln, Lincolnshire, United Kingdom
| | - Bethany Growns
- School of Psychology, University of New South Wales, Sydney, New South Wales, Australia
- School of Social and Behavioural Sciences, Arizona State University, Tempe, Arizona, United States of America
| | - An Yan
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Yuqing Wang
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Kun Guo
- School of Psychology, University of Lincoln, Lincoln, Lincolnshire, United Kingdom
| | - Robin S. S. Kramer
- School of Psychology, University of Lincoln, Lincoln, Lincolnshire, United Kingdom
| | - Gary Edmond
- Faculty of Law and Justice, University of New South Wales, Sydney, New South Wales, Australia
| | - Kristy A. Martire
- School of Psychology, University of New South Wales, Sydney, New South Wales, Australia
| | - Mehera San Roque
- Faculty of Law and Justice, University of New South Wales, Sydney, New South Wales, Australia
| | - David White
- School of Psychology, University of New South Wales, Sydney, New South Wales, Australia
| |
Collapse
|
22
|
Ethics of Smart Cities: Towards Value-Sensitive Design and Co-Evolving City Life. SUSTAINABILITY 2021. [DOI: 10.3390/su132011162] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The digital revolution has brought about many societal changes such as the creation of “smart cities”. The smart city concept has changed the urban ecosystem by embedding digital technologies in the city fabric to enhance the quality of life of its inhabitants. However, it has also led to some pressing issues and challenges related to data, privacy, ethics inclusion, and fairness. While the initial concept of smart cities was largely technology- and data-driven, focused on the automation of traffic, logistics and processes, this concept is currently being replaced by technology-enabled, human-centred solutions. However, this is not the end of the development, as there is now a big trend towards “design for values”. In this paper, we point out how a value-sensitive design approach could promote a more sustainable pathway of cities that better serves people and nature. Such “value-sensitive design” will have to take ethics, law and culture on board. We discuss how organising the digital world in a participatory way, as well as leveraging the concepts of self-organisation, self-regulation, and self-control, would foster synergy effects and thereby help to leverage a sustainable technological revolution on a global scale. Furthermore, a “democracy by design” approach could also promote resilience.
Collapse
|
23
|
Abstract
Deep learning models currently achieve human levels of performance on real-world face recognition tasks. We review scientific progress in understanding human face processing using computational approaches based on deep learning. This review is organized around three fundamental advances. First, deep networks trained for face identification generate a representation that retains structured information about the face (e.g., identity, demographics, appearance, social traits, expression) and the input image (e.g., viewpoint, illumination). This forces us to rethink the universe of possible solutions to the problem of inverse optics in vision. Second, deep learning models indicate that high-level visual representations of faces cannot be understood in terms of interpretable features. This has implications for understanding neural tuning and population coding in the high-level visual cortex. Third, learning in deep networks is a multistep process that forces theoretical consideration of diverse categories of learning that can overlap, accumulate over time, and interact. Diverse learning types are needed to model the development of human face processing skills, cross-race effects, and familiarity with individual faces.
Collapse
Affiliation(s)
- Alice J O'Toole
- School of Behavioral and Brain Sciences, The University of Texas at Dallas, Richardson, Texas 75080, USA;
| | - Carlos D Castillo
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, Maryland 21218, USA;
| |
Collapse
|
24
|
Henderson M, Serences JT. Biased orientation representations can be explained by experience with nonuniform training set statistics. J Vis 2021; 21:10. [PMID: 34351397 PMCID: PMC8354037 DOI: 10.1167/jov.21.8.10] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Visual acuity is better for vertical and horizontal compared to other orientations. This cross-species phenomenon is often explained by “efficient coding,” whereby more neurons show sharper tuning for the orientations most common in natural vision. However, it is unclear if experience alone can account for such biases. Here, we measured orientation representations in a convolutional neural network, VGG-16, trained on modified versions of ImageNet (rotated by 0°, 22.5°, or 45° counterclockwise of upright). Discriminability for each model was highest near the orientations that were most common in the network's training set. Furthermore, there was an overrepresentation of narrowly tuned units selective for the most common orientations. These effects emerged in middle layers and increased with depth in the network, though this layer-wise pattern may depend on properties of the evaluation stimuli used. Biases emerged early in training, consistent with the possibility that nonuniform representations may play a functional role in the network's task performance. Together, our results suggest that biased orientation representations can emerge through experience with a nonuniform distribution of orientations, supporting the efficient coding hypothesis.
Collapse
Affiliation(s)
- Margaret Henderson
- Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA, USA.,Department of Machine Learning, Carnegie Mellon University, Pittsburgh, PA, USA.,Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA.,
| | - John T Serences
- Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA, USA.,Department of Psychology, University of California, San Diego, La Jolla, CA, USA.,Kavli Foundation for the Brain and Mind, University of California, San Diego, La Jolla, CA, USA.,
| |
Collapse
|
25
|
Zhou X, Burton AM, Jenkins R. Two Factors in Face Recognition: Whether You Know the Person's Face and Whether You Share the Person's Race. Perception 2021; 50:524-539. [PMID: 33983068 PMCID: PMC8371284 DOI: 10.1177/03010066211014016] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
One of the best-known phenomena in face recognition is the other-race effect, the observation that own-race faces are better remembered than other-race faces. However, previous studies have not put the magnitude of other-race effect in the context of other influences on face recognition. Here, we compared the effects of (a) a race manipulation (own-race/other-race face) and (b) a familiarity manipulation (familiar/unfamiliar face) in a 2 × 2 factorial design. We found that the familiarity effect was several times larger than the race effect in all performance measures. However, participants expected race to have a larger effect on others than it actually did. Face recognition accuracy depends much more on whether you know the person’s face than whether you share the same race.
Collapse
|
26
|
Boosting Fraud Detection in Mobile Payment with Prior Knowledge. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11104347] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
With the prevalence of mobile e-commerce, fraudulent transactions conducted by robots are becoming increasingly common in mobile payments, which is severely undermining market fairness and resulting in financial losses. It has become a difficult problem for mobile applications to identify robotic automation accurately and efficiently from a massive number of transactions. The current research does not propose any effective method or engineering implementation. In this article, an extension to boost algorithms is presented that permits the incorporation of prior human knowledge as a means of compensating for a training data shortage and improving prediction results. Prior human knowledge is accumulated from historical fraud transactions or transferred from different domains in the form of expert rules and blacklists. The knowledge is applied to extract risk features from transaction data, risk features together with normal features are input into the boosting algorithm to perform training, and therefore we incorporate boosting algorithm with prior human knowledge to improve the performance of the model. For the first time we verified the effectiveness of the method via a widely deployed mobile APP with 150+ million users, and by taking experiments on a certain dataset, the extended boosting model shows an accuracy increase from 0.9825 to 0.9871 and a recall rate increase from 0.888 to 0.948. We also investigated feature differences between robots and normal users and we discovered the behavior patterns of robotic automation that include less spatial motion detected by device sensors (1/10 of normal user pattern), higher IP group-clustering ratio (60% in robots vs. 15% in normal users), higher jailbroken device rate (92.47% vs. 4.64%), more irregular device names and fewer IP address changes. The quantitative analysis result is helpful for APP developers and service providers to understand and prevent fraudulent transactions from robotic automation.This article proposed an optimized boosting model, which has better use in the field of robotic automation detection of mobile phones. By combining prior knowledge and feature importance analysis, the model is more robust when the actual dataset is unbalanced or with few-short samples. The model is also more explainable as feature analysis is available which can be used for generating disposal rules in the actual fake mobile user blocking systems.
Collapse
|
27
|
Educational Warehouse: Modular, Private and Secure Cloudable Architecture System for Educational Data Storage, Analysis and Access. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11020806] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Data in the educational context are becoming increasingly important in decision-making and teaching-learning processes. Similar to the industrial context, educational institutions are adopting data-processing technologies at all levels. To achieve representative results, the processes of extraction, transformation and uploading of educational data should be ubiquitous because, without useful data, either internal or external, it is difficult to perform a proper analysis and to obtain unbiased educational results. It should be noted that the source and type of data are heterogeneous and that the analytical processes can be so diverse that it opens up a practical problem of management and access to the data generated. At the same time, ensuring the privacy, identity, confidentiality and security of students and their data is a “sine qua non” condition for complying with the legal issues involved while achieving the required ethical premises. This work proposes a modular and scalable data system architecture that solves the complexity of data management and access. On the one hand, it allows educational institutions to collect any data generated in both the teaching-learning and management processes. On the other hand, it will enable external access to this data under appropriate privacy and security conditions.
Collapse
|