Forensic facial comparison in South Africa: State of the science

Post-mortem to ante-mortem facial image comparison for deceased migrant identification

The identification of deceased migrants is a global challenge that is exacerbated by migration distance, post-mortem conditions, access to ante-mortem data for comparison, inconsistent international procedures and lack of communication between arrival and origin countries. Due to low technology requirements, fast speed analysis and ease of transferring digital data, facial image comparison is particularly beneficial in those contexts, especially in challenging scenarios when this may be the only initial ante-mortem data available to identify the deceased. The Facial Identification Scientific Working Group (FISWG) professional guidelines for facial image comparison were developed for living facial appearance, and, therefore, a tailored protocol for the application of post-mortem to ante-mortem facial image comparison was proposed and evaluated in this research. The protocol was investigated via an inter-observer and an accuracy study, using 29 forensic cases (2001-2020) from the University of Milan, provided by the Laboratory of Forensic Anthropology and Odontology. In order to replicate a migrant identification scenario, each post-mortem subject was compared to all 29 ante-mortem targets (841 comparisons). The protocol guided the practitioner through stages of facial image comparison, from broad (phase 1) to more detailed (phase 3), eventually leading to a decision of 'exclusion' or 'potential match' for each post-mortem to ante-mortem case (phase 4). In phase 4, a support scale was also utilised to indicate the level of confidence in a potential match. Each post-mortem subject could be recorded with multiple potential matches. The protocol proved to be useful guide for facial image comparison, especially for less experienced practitioners and the inter-observer study suggested good reproducibility. The majority (82-96%) of ante-mortem subjects were excluded at the first stage of the protocol, and 71 full post-mortem to ante-mortem facial image comparisons were carried out. On average, two or three potential matches were recorded for each post-mortem subject. The overall accuracy rate was 85%, with the majority (79%) of ante-mortem non-targets correctly excluded from the identification process. An increased number and quality of available ante-mortem images produced more successful matches with higher levels of support. All potential matches involving non-targets received low levels of support, and for 73% of the post-mortem subjects, the ante-mortem target was the only recorded potential match. However, two ante-mortem targets were incorrectly excluded (one at the first stage of the protocol) and therefore changes to the protocol were implemented to mitigate these errors. A full protocol and a practical recording chart for practitioner use is included with this paper.

Head poses and grimaces: Challenges for automated face identification algorithms?

In today's biometric and commercial settings, state-of-the-art image processing relies solely on artificial intelligence and machine learning which provides a high level of accuracy. However, these principles are deeply rooted in abstract, complex "black-box systems". When applied to forensic image identification, concerns about transparency and accountability emerge. This study explores the impact of two challenging factors in automated facial identification: facial expressions and head poses. The sample comprised 3D faces with nine prototype expressions, collected from 41 participants (13 males, 28 females) of European descent aged 19.96 to 50.89 years. Pre-processing involved converting 3D models to 2D color images (256 × 256 px). Probes included a set of 9 images per individual with head poses varying by 5° in both left-to-right (yaw) and up-and-down (pitch) directions for neutral expressions. A second set of 3,610 images per individual covered viewpoints in 5° increments from -45° to 45° for head movements and different facial expressions, forming the targets. Pair-wise comparisons using ArcFace, a state-of-the-art face identification algorithm yielded 54,615,690 dissimilarity scores. Results indicate that minor head deviations in probes have minimal impact. However, the performance diminished as targets deviated from the frontal position. Right-to-left movements were less influential than up and down, with downward pitch showing less impact than upward movements. The lowest accuracy was for upward pitch at 45°. Dissimilarity scores were consistently higher for males than for females across all studied factors. The performance particularly diverged in upward movements, starting at 15°. Among tested facial expressions, happiness and contempt performed best, while disgust exhibited the lowest AUC values.

Focus distance estimation from photographed faces: a test of PerspectiveX using 1709 frontal and profile photographs from DSLR and smartphone cameras

As focus distance (FD) sets perspective, it is an important consideration for the forensic analysis of faces in photographs, including those used for craniofacial superimposition. In the craniofacial superimposition domain, the PerspectiveX algorithm has been suggested for FD estimation. This algorithm uses a mean value of palpebral fissure length, as a scale, to estimate the FD. So far, PerspectiveX has not been validated for profile view photographs or for photographs taken with smartphones. This study tests PerspectiveX in both front and profile views, using multiple DSLR cameras, lenses and smartphones. In total, 1709 frontal and 1709 profile photographs of 10 adult participants were tested at 15 ground truth FDs using three DSLR cameras with 12 camera/lens combinations, five smartphone back cameras and four smartphone front cameras. Across all distances, PerspectiveX performed with a mean absolute error (MAE) of 11% and 12% for DSLR photographs in frontal and profile views, respectively, while errors doubled for frontal and profile photographs from smartphones (26% and 27%, respectively). This reverifies FD estimation for frontal DSLR photographs, validates FD estimates from profile view DSLR photographs and shows that FD estimation is currently inaccurate for smartphones. Until such time that FD estimations for facial photographs taken using smartphones improves, DSLR or 35 mm film images should continue to be sought for craniofacial superimpositions.

Forensic Facial Comparison: Current Status, Limitations, and Future Directions

Global escalation of crime has necessitated the use of digital imagery to aid the identification of perpetrators. Forensic facial comparison (FFC) is increasingly employed, often relying on poor-quality images. In the absence of standardized criteria, especially in terms of video recordings, verification of the methodology is needed. This paper addresses aspects of FFC, discussing relevant terminology, investigating the validity and reliability of the FISWG morphological feature list using a new South African database, and advising on standards for CCTV equipment. Suboptimal conditions, including poor resolution, unfavorable angle of incidence, color, and lighting, affected the accuracy of FFC. Morphological analysis of photographs, standard CCTV, and eye-level CCTV showed improved performance in a strict iteration analysis, but not when using analogue CCTV images. Therefore, both strict and lenient iterations should be conducted, but FFC must be abandoned when a strict iteration performs worse than a lenient one. This threshold ought to be applied to the specific CCTV equipment to determine its utility. Chance-corrected accuracy was the most representative measure of accuracy, as opposed to the commonly used hit rate. While the use of automated systems is increasing, trained human observer-based morphological analysis, using the FISWG feature list and an Analysis, Comparison, Evaluation, and Verification (ACE-V) approach, should be the primary method of facial comparison.

Performance of forensic facial comparison by morphological analysis across optimal and suboptimal CCTV settings

Facial comparison is an important yet understudied discipline in forensics. The recommended method for facial comparison in a forensic setting involves morphological analysis (MA) with the use of a facial feature list. The performance of this approach has not been tested across various closed-circuit television (CCTV) conditions. This is of particular concern as video and image data available to law enforcement is often varied and of subpar conditions. The present study aimed at testing MA across two types of CCTV data, representing ideal and less than ideal settings, also assessing which particular shortcomings arose from less-than-ideal settings. The study was conducted on a subset of the Wits Face Database arranged in a total of 225 face pools. Each face pool consisted of a target image obtained from either a high-definition digital CCTV camera or a low-definition analogue CCTV camera in monochrome, contrasted to 10 possible matches. The face pools were analysed and scored using MA and confusion matrices were used to analyse the outcomes. A notably high chance corrected accuracy (CCA) (97.3%) and reliability (0.969) was identified across the digital CCTV sample, while in the analogue CCTV sample MA appeared to underperform both in accuracy (CCA: 33.1%) and reliability (0.529). The majority of the errors in scoring resulted in false negatives in the analogue sample (75.2%), while across both CCTV conditions false positives were low (digital: 0.3%; analogue: 1.2%). Even though hit rates appeared deceptively high in the analogue sample, the various measures of performance used and particularly the chance corrected accuracy highlighted its shortfalls. Overall, CCTV recording quality appears closely associated to MA performance, despite the favourable error rates when using the Facial Identification Scientific Working Group feature list.

Assessing the effect of facial disguises on forensic facial comparison by morphological analysis

Disguises are commonly used to mask a person's facial appearance in areas under closed-circuit television (CCTV) surveillance. While many studies attempted to understand the effects of disguises, such as hats and glasses, on facial recognition, limited studies have looked at disguises in forensic facial comparison. The aim of this study was to compare the outcomes of forensic facial comparison by morphological analysis (MA) in a CCTV sample with sunglasses and brimmed caps. The sample was obtained from the Wits Face Database and organized into 81 face pools of one target facial image wearing a disguise (cap or sunglasses) and 10 potential matching images. MA was conducted across face pools, and confusion matrices were used to assess the outcomes. Surprisingly, sunglasses had limited effect on MA performance both in accuracy (90.4%) and in reliability (κ = 0.798), while caps markedly decreased both accuracy (68.1%) and reliability (κ = 0.639). Error rates were associated primarily with false negatives in both samples (caps: 42.4%; sunglasses: 16.1%) despite the sample distribution favoring false-positive errors, which were very low (caps: 0.6%; sunglasses: 0%). Similarly to other studies, hats and caps were more harmful to correct identification when compared to sunglasses, which actually resulted in better accuracy than regular CCTV recordings. The effect of brimmed caps on accuracy was attributed to the overall loss of facial information caused. On training analysts, it may be helpful to instruct purposefully avoiding overreliance on easily disguised facial features, as other regions of the face also contain substantial feature information.

Development of the Wits Face Database: an African database of high-resolution facial photographs and multimodal closed-circuit television (CCTV) recordings

Forensic facial comparison is a commonly used, yet under-evaluated method employed in medicolegal contexts across the world. Testing the accuracy and reliability of facial comparisons requires large scale controlled and matching facial image databases. Databases that contain images of individuals on closed-circuit television (CCTV), with matching formal and informal photographs are needed for this type of research. Although many databases are available, the majority if not all are developed in order to improve facial recognition and face detection algorithms through machine learning, with very limited if any measure of standardisation. This paper aims to review the available databases and describe the development of a high resolution, standardised facial photograph and CCTV recording database of male Africans. The database is composed of a total of 6220 standardised and uncontrolled suboptimal facial photographs of 622 matching individuals in five different views, as well as corresponding CCTV footage of 334 individuals recorded under different realistic conditions. A detailed description of the composition and acquisition process of the database as well as its subdivisions and possible uses are provided. The challenges and limitations of developing this database are also highlighted, particularly with regard to obtaining CCTV video recordings and ethics for a database of faces. The application process to access the database is also briefly described.

Validation of forensic facial comparison by morphological analysis in photographic and CCTV samples

Between the ever-increasing availability of surveillance evidence and expert-based forensic facial comparison being considered admissible in court, confirming its validity is paramount. Facial comparison is most commonly conducted using morphological analysis (MA), a largely untested feature-based approach. This study aimed at validating the current recommended practice of MA in both standardised and suboptimal surveillance samples. Face pools of 175 South African males were compiled with a series of facial photographs, using images from the Wits Face Database. The first 75 face pools consisted of wildtype (unstandardised) high-quality target photographs, while the remaining 100 face pools consisted of suboptimal closed-circuit television (CCTV) target images. Target images were compared to high-quality standardised photographs. Face pools were analysed using the Facial Identification Scientific Working Group's guidelines and feature list. Confusion matrices were used to determine the performance of MA in each cohort. MA was found highly accurate (chance-corrected accuracy (CCA): 99.1%) and reliable (κ = 0.921) in the photographic sample and less accurate (CCA: 82.6%) and reliable (κ = 0.743), in the CCTV sample. Higher false-positive and false-negative rates were noted for the CCTV sample, with the majority of errors resulting in false-negative outcomes. The decreased performance in the CCTV sample was attributed to various factors including image quality, angle of recording and lighting. Other studies testing facial comparison identified lower accuracies and reliability across various conditions. Better performance was found here and in other studies that included some form of facial feature list, reinforcing the importance of using a systematic facial feature list.

An overview of the latest developments in facial imaging

Facial imaging is a term used to describe methods that use facial images to assist or facilitate human identification. This pertains to two craniofacial identification procedures that use skulls and faces-facial approximation and photographic superimposition-as well as face-only methods for age progression/regression, the construction of facial graphics from eyewitness memory (including composites and artistic sketches), facial depiction, face mapping and newly emerging methods of molecular photofitting. Given the breadth of these facial imaging techniques, it is not surprising that a broad array of subject-matter experts participate in and/or contribute to the formulation and implementation of these methods (including forensic odontologists, forensic artists, police officers, electrical engineers, anatomists, geneticists, medical image specialists, psychologists, computer graphic programmers and software developers). As they are concerned with the physical characteristics of humans, each of these facial imaging areas also falls in the domain of physical anthropology, although not all of them have been traditionally regarded as such. This too offers useful opportunities to adapt established methods in one domain to others more traditionally held to be disciplines within physical anthropology (e.g. facial approximation, craniofacial superimposition and face photo-comparison). It is important to note that most facial imaging methods are not currently used for identification but serve to assist authorities in narrowing or directing investigations such that other, more potent, methods of identification can be used (e.g. DNA). Few, if any, facial imaging approaches can be considered honed end-stage scientific methods, with major opportunities for physical anthropologists to make meaningful contributions. Some facial imaging methods have considerably stronger scientific underpinnings than others (e.g. facial approximation versus face mapping), some currently lie entirely within the artistic sphere (facial depiction), and yet others are so aspirational that realistic capacity to obtain their aims has strongly been questioned despite highly advanced technical approaches (molecular photofitting). All this makes for a broad-ranging, dynamic and energetic field that is in a constant state of flux. This manuscript provides a theoretical snapshot of the purposes of these methods, the state of science as it pertains to them, and their latest research developments.