Getting Rid of Dichotomous Sex Estimations: Why Logistic Regression Should be Preferred Over Discriminant Function Analysis

Classification performance of the Sella-Tunis et al. (2017) sex estimation method in Czech population: different posterior probability threshold approaches

In this study we tested classification performance of a sex estimation method from the mandible originally developed by Sella-Tunis et al. (2017) on a heterogeneous Israeli population. Mandibular linear dimensions were measured on 60 CT scans derived from the Czech living population. Classification performance of Israeli discriminant functions (DFs-IL) was analyzed in comparison with calculated Czech discriminant functions (DFs-CZ) while different posterior probability thresholds (currently discussed in the forensic literature) were employed. Our results comprehensively illustrate sensitivity of different discriminant functions to population differences in body size and degree of sexual dimorphism. We demonstrate that the error rate may be biased when presented per posterior probability threshold. DF-IL 1 showed least sensitivity to population origin and fulfilled criteria of sufficient classification performance when applied on the Czech sample with a minimum posterior probability threshold of 0.88 reaching overall accuracy ≥ 95%, zero sex bias, and 80% of classified individuals. The last parameter was higher in DF-CZ 1 which was the main difference between those two DFs suggesting relatively low dependance on population origin. As the use of population-specific methods is often prevented by complicated assessment of population origin, DF-IL 1 is a candidate for a sufficiently robust method that could be reliably applied outside the reference sample, and thus, its classification performance deserves further testing on more population samples.

Utilising Discriminant Function Analysis (DFA) for Classifying Osteoarthritis (OA) Patients and Volunteers Based on Biomarker Concentration

Osteoarthritis (OA) is a degenerative joint disease characterised by the breakdown of cartilage, causing pain, stiffness, and limited movement. Early diagnosis is crucial for effective management but remains challenging due to non-specific early symptoms. This study explores the application of Discriminant Function Analysis (DFA) to classify OA patients and healthy volunteers based on biomarker concentrations of Interleukin-6 (IL-6), Tumour necrosis factor-alpha (TNF-α), and Myeloperoxidase (MPO). DFA was employed to analyse biomarker data from 86 participants (58 patients, 28 volunteers) to evaluate the discriminatory power of these biomarkers in predicting OA. Significant differences were observed in MPO and TNF-α levels between groups, while IL-6 did not show a significant distinction. The iterative classification process improved model assumptions and classification accuracy, achieving a pre-classification accuracy of 71.8%, which adjusted to 57.1% post-classification. The results highlight DFA's potential in OA diagnosis, suggesting its utility in managing complex data and aiding personalised treatment strategies. The study underscores the need for larger sample sizes and additional biomarkers to enhance diagnostic robustness and provides a foundation for integrating DFA into clinical practice for early OA detection.

Prioritizing a high posterior probability threshold leading to low error rate over high classification accuracy: the validity of MorphoPASSE software for cranial morphological sex estimation in a contemporary population

An increasing number of software tools can be used in forensic anthropology to estimate a biological profile, but further studies in other populations are required for more robust validation. The present study aimed to evaluate the validity of MorphoPASSE software for sex estimation from sexually dimorphic cranial traits recorded on 3D CT models (n = 180) from three populations samples (Czech, French, and Egyptian). Two independent observers performed scoring of 4 cranial traits (2 of them bilateral) in each population sample of 30 males and 30 females. The accuracy of sex estimation using traditional posterior probability threshold (pp = 0.5) ranged from 85.6% to 88.3% and overall classification error from 14.4% to 11.7% for both observers, and corresponds to the previously published values of the method. The MorphoPASSE method is also affected by the subjectivity of the observers, as both observers show agreement in sex assignment in 83.9% of cases, regardless of the accuracy of the estimates. Applying a higher posterior probability threshold (pp 0.95) provided classification accuracy of 97.9% and 93.3% of individuals (for observer A and B respectively), minimizing the risk of error to 2.1% and 6.7%, respectively. However, sex estimation can only be applied to 54% and 66% of individuals, respectively. Our results demonstrate the validity of the MorphoPASSE software for cranial sex estimation outside the reference population. However, the achieved classification success is accompanied by a high risk of errors, the reduction of which is only possible by increasing the posterior probability threshold.

Estimation of sex using dimensions around the metatarsal diaphyseal nutrient foramen: Application of discriminant function analysis and logistic regression models

Sex estimation equations are population-specific, and a wider use of multiple bones to generate equations will increase the accuracy of sex estimation in forensic settings. The metatarsal bones have been used previously, however the dimensions around the diaphyseal nutrient foramen have not been utilised in sex estimation. The current study aimed to determine the utility of the dimensions around the nutrient foramen of metatarsal bones in estimating sex in the South Africans of European descent (SAED). Five measurements around the nutrient foramen were taken from a total of 876 metatarsal bones (first to fifth) from 186 individual skeletons (99 males, 87 females) obtained from the Raymond A. Dart Modern Skeletal Collection. Measurements subjected to direct and stepwise discriminant function (DFA) and logistic regression (LRA) analyses included total length, distance from proximal end to nutrient foramen, circumference, and mediolateral and dorsoplantar diameters at the level of the nutrient foramen. The original classification accuracies for multivariable functions of the stepwise and direct DFA ranged from 83.1-88.3% to 85.5-88.3%, respectively. The original classification accuracies for multivariable functions of the stepwise and direct LRA ranged from 83.3%-88.7% to 86.2%-88.3%, respectively. The cross-validation classifications showed a drop of 0-2.4% for DFA and 0.2-1.1% for LRA. The width measurements were better predictors of sex than length. The dimensions around the metatarsal bone nutrient foramen exhibit sexual dimorphism in the SAED. The generated DFA and LRA functions produced high average classification accuracies which are useful in sex estimation during forensic human identification.

An overview of log likelihood ratio cost in forensic science - Where is it used and what values can we expect?

There is increasing support for reporting evidential strength as a likelihood ratio (LR) and increasing interest in (semi-)automated LR systems. The log-likelihood ratio cost (Cllr) is a popular metric for such systems, penalizing misleading LRs further from 1 more. Cllr = 0 indicates perfection while Cllr = 1 indicates an uninformative system. However, beyond this, what constitutes a "good" Cllr is unclear. Aiming to provide handles on when a Cllr is "good", we studied 136 publications on (semi-)automated LR systems. Results show Cllr use heavily depends on the field, e.g., being absent in DNA analysis. Despite more publications on automated LR systems over time, the proportion reporting Cllr remains stable. Noticeably, Cllr values lack clear patterns and depend on the area, analysis and dataset. As LR systems become more prevalent, comparing them becomes crucial. This is hampered by different studies using different datasets. We advocate using public benchmark datasets to advance the field.

Assigned sex estimation with the clavicle and scapula: A study in a Portuguese reference sample

The estimation of biological sex is a critical step in the assessment of the biological profile of an anonymous skeletonized individual. In certain recovery circumstances, the most dimorphic skeletal areas, such as the pelvis, are absent or fragmented; in that case, other bones of the skeleton, including the clavicle and scapula, can be used to predict sex. The purpose of this research is to generate new models for the estimation of sex with clavicular and scapular measurements using a study-sample of 129 individuals with clavicle (65 males and 64 females) and 112 individuals with scapula (50 males and 62 females) from the Lisbon Identified Skeletal Collection (Portugal). A decision tree classifier (C4.5) and logistic regression (LR) were employed to create univariable and multivariable sex prediction models. Accuracy under cross-validation of the classification models is high (up to 93.8%), with minimal bias (<5%), particularly in the multivariable models. The proposed LR models facilitate the probabilistic estimation of biological sex, accounting for the significant overlap in the expression of sexual dimorphism.

Sex estimation using measurements of the proximal femur in a historical population from Poland

Sex estimation is one of the most important components in assessing the biological profile of an individual. In an archaeological context, the pelvis, which is the most dimorphic part of the skeleton, is often poorly preserved, which can cause an inability to use morphological sex estimation methods. Therefore, alternative methods are required in such cases. Because the utility of the metric methods based on the femur measurement has been confirmed, and the bone is usually available for examination due to its good preservation, developing methods using the landmarks of the femur could have important implications in sex estimation.   This study aimed to derive a discriminant function equation for a Polish archaeological population based on measurements of the proximal end of the femur.   The study sample included individuals from a medieval cemetery in Milicz (n = 62) and an early modern necropolis at Czysty Square in Wrocław, Poland (n = 162). The analysis included seven measurements collected from the right and left proximal femora. To estimate the reproducibility of the measurements, intra-and interobserver errors and reliability coefficients were calculated. Subsequently, univariate and stepwise discriminant analyses were performed, and the sex sectioning points and equations were proposed.   No differences were observed between measurements of the right and left femora. The results indicated a high utility and reproducibility of the FHD measurement (regardless whether left or right femur was measured). The discriminant equations for sex estimation reached an accuracy of 83.0–92.3%, which implies the utility of the function on Polish historical populations when the other methods for sex estimation cannot be used. 

Efficiency of the Adjusted Binary Classification (ABC) Approach in Osteometric Sex Estimation: A Comparative Study of Different Linear Machine Learning Algorithms and Training Sample Sizes

The adjusted binary classification (ABC) approach was proposed to assure that the binary classification model reaches a particular accuracy level. The present study evaluated the ABC for osteometric sex classification using multiple machine learning (ML) techniques: linear discriminant analysis (LDA), boosted generalized linear model (GLMB), support vector machine (SVM), and logistic regression (LR). We used 13 femoral measurements of 300 individuals from a modern Turkish population sample and split data into two sets: training (n = 240) and testing (n = 60). Then, the five best-performing measurements were selected for training univariate models, while pools of these variables were used for the multivariable models. ML classifier type did not affect the performance of unadjusted models. The accuracy of univariate models was 82−87%, while that of multivariate models was 89−90%. After applying ABC to the crossvalidation set, the accuracy and the positive and negative predictive values for uni- and multivariate models were ≥95%. Sex could be estimated for 28−75% of individuals using univariate models but with an obvious sexing bias, likely caused by different degrees of sexual dimorphism and between-group overlap. However, using multivariate models, we minimized the bias and properly classified 81−87% of individuals. A similar performance was also noted in the testing sample (except for FEB), with accuracies of 96−100%, and a proportion of classified individuals between 30% and 82% in univariate models, and between 90% and 91% in multivariate models. When considering different training sample sizes, we demonstrated that LR was the most sensitive with limited sample sizes (n < 150), while GLMB was the most stable classifier.

Performance of the supervised learning algorithms in sex estimation of the proximal femur: A comparative study in contemporary Egyptian and Turkish samples

Sex estimation standards are population specific however, we argue that machine learning techniques (ML) may enhance the biological sex determination on trans-population application. Linear discriminant analysis (LDA) versus nine ML including quadratic discriminant analysis (QDA), support vector machine (SVM), Decision Tree (DT), Gaussian process (GPC), Naïve Bayesian (NBC), K-Nearest Neighbor (KNN), Random Forest (RFM) and Adaptive boosting (Adaboost) were compared. The experiments involve two contemporary populations: Turkish (n = 300) and Egyptian populations (n = 100) for training and validation, respectively. Base models were calibrated using isotonic and sigmoid calibration schemes. Results were analyzed at posterior probabilities (pp) thresholds >0.95 and >0.80. At pp = 0.5, ML algorithms yielded comparable accuracies in the training (90% to 97%) and test sets (81% to 88%) which are not modified after employing the calibration techniques. At pp >0.95, the raw RFM, LDA, QDA, and SVM models have shown the best performance however, calibration techniques improved the performance of various classifier especially NBC and Adaboost. By contrast, the performance of GPC, KNN, QDA models worsened by calibration. RFM has shown the best performance among all models at both thresholds whereas LDA benefited the best from using both calibration methods at pp >0.80. Complex ML models are not necessarily achieving better performance metrics. LDA and QDA remain the fastest and simplest classifiers. We demonstrated the capability of enhancing sex estimation using ML on an independent population sample however, differences in the underlying probability distribution generated by models were detected which warranted more cautious application by forensic practitioners.

Forensic Tools for Species Identification of Skeletal Remains: Metrics, Statistics, and OsteoID

Although nonhuman remains constitute a significant portion of forensic anthropological casework, the potential use of bone metrics to assess the human origin and to classify species of skeletal remains has not been thoroughly investigated. This study aimed to assess the utility of quantitative methods in distinguishing human from nonhuman remains and present additional resources for species identification. Over 50,000 measurements were compiled from humans and 27 nonhuman (mostly North American) species. Decision trees developed from the long bone data can differentiate human from nonhuman remains with over 90% accuracy (>98% accuracy for the human sample), even if all long bones are pooled. Stepwise discriminant function results were slightly lower (>87.4% overall accuracy). The quantitative models can be used to support visual identifications or preliminarily assess forensic significance at scenes. For species classification, bone-specific discriminant functions returned accuracies between 77.7% and 89.1%, but classification results varied highly across species. From the study data, we developed a web tool, OsteoID, for users who can input measurements and be shown photographs of potential bones/species to aid in visual identification. OsteoID also includes supplementary images (e.g., 3D scans), creating an additional resource for forensic anthropologists and others involved in skeletal species identification and comparative osteology.

Calculation of likelihood ratios for inference of biological sex from human skeletal remains

It is common in forensic anthropology to draw inferences (e.g., inferences with respect to biological sex of human remains) using statistical models applied to anthropometric data. Commonly used models can output posterior probabilities, but a threshold is usually applied in order to obtain a classification. In the forensic-anthropology literature, there is some unease with this "fall-off-the-cliff" approach. Proposals have been made to exclude results that fall within a "zone of uncertainty", e.g., if the posterior probability for "male" is greater than 0.95 then the remains are classified as male, and if the posterior probability for "male" is less than 0.05 then the remains are classified as female, but if the posterior probability for "male" is between 0.05 and 0.95 the remains are not classified as either male or female. In the present paper, we propose what we believe is a simpler solution that is in line with interpretation of evidence in other branches of forensic science: implementation of the likelihood-ratio framework using relevant data, quantitative measurements, and statistical models. Statistical models that can implement this approach are already widely used in forensic anthropology. All that is required are minor modifications in the way those models are used and a change in the way practitioners and researchers think about the meaning of the output of those models. We explain how to calculate likelihood ratios using osteometric data and linear discriminant analysis, quadratic discriminant analysis, and logistic regression models. We also explain how to empirically validate likelihood-ratio models.

Revisiting metric sex estimation of burnt human remains via supervised learning using a reference collection of modern identified cremated individuals (Knoxville, USA)

OBJECTIVES

This study aims to increase the rate of correctly sexed calcined individuals from archaeological and forensic contexts. This is achieved by evaluating sexual dimorphism of commonly used and new skeletal elements via uni- and multi-variate metric trait analyses.

MATERIALS AND METHODS

Twenty-two skeletal traits were evaluated in 86 individuals from the William M. Bass donated cremated collection of known sex and age-at-death. Four different predictive models, logistic regression, random forest, neural network, and calculation of population specific cut-off points, were used to determine the classification accuracy (CA) of each feature and several combinations thereof.

RESULTS

An overall CA of ≥ 80% was obtained for 12 out of 22 features (humerus trochlea max., and lunate length, humerus head vertical diameter, humerus head transverse diameter, radius head max., femur head vertical diameter, patella width, patella thickness, and talus trochlea length) using univariate analysis. Multivariate analysis showed an increase of CA (≥ 95%) for certain combinations and models (e.g., humerus trochlea max. and patella thickness). Our study shows metric sexual dimorphism to be well preserved in calcined human remains, despite the changes that occur during burning.

CONCLUSIONS

Our study demonstrated the potential of machine learning approaches, such as neural networks, for multivariate analyses. Using these statistical methods improves the rate of correct sex estimations in calcined human remains and can be applied to highly fragmented unburnt individuals from both archaeological and forensic contexts.