Four model variants within a continuous forensic DNA mixture interpretation framework: Effects on evidential inference and reporting

A collaborative study on the precision of the Markov chain Monte Carlo algorithms used for DNA profile interpretation

Several fully continuous probabilistic genotyping software (PGS) use Markov chain Monte Carlo algorithms (MCMC) to assign weights to different proposed genotype combinations at a locus. Replicate interpretations of the same profile in these software are expected not to produce identical weights and likelihood ratio (LR) values due to the Monte Carlo aspect. This paper reports a detailed precision study under reproducibility conditions conducted as a collaborative exercise across the National Institute of Standards and Technology (NIST), Federal Bureau of Investigation (FBI), and Institute of Environmental Science and Research (ESR). Replicate interpretations generated across the three laboratories used the same input files, software version, and settings but different random number seed and different computers. This work demonstrates that using different computers to analyze replicate interpretations does not contribute to any variations in LR values. The study quantifies the magnitude of differences in the assigned LRs that is only due to run-to-run MCMC variability and addresses the potential explanations for the observed differences.

An overview of log likelihood ratio cost in forensic science - Where is it used and what values can we expect?

There is increasing support for reporting evidential strength as a likelihood ratio (LR) and increasing interest in (semi-)automated LR systems. The log-likelihood ratio cost (Cllr) is a popular metric for such systems, penalizing misleading LRs further from 1 more. Cllr = 0 indicates perfection while Cllr = 1 indicates an uninformative system. However, beyond this, what constitutes a "good" Cllr is unclear. Aiming to provide handles on when a Cllr is "good", we studied 136 publications on (semi-)automated LR systems. Results show Cllr use heavily depends on the field, e.g., being absent in DNA analysis. Despite more publications on automated LR systems over time, the proportion reporting Cllr remains stable. Noticeably, Cllr values lack clear patterns and depend on the area, analysis and dataset. As LR systems become more prevalent, comparing them becomes crucial. This is hampered by different studies using different datasets. We advocate using public benchmark datasets to advance the field.

Decreased accuracy of forensic DNA mixture analysis for groups with lower genetic diversity

Forensic investigation of DNA samples from multiple contributors has become commonplace. These complex analyses use statistical frameworks accounting for multiple levels of uncertainty in allelic contributions from different individuals, particularly for samples containing few molecules of DNA. These methods have been thoroughly tested along some axes of variation, but less attention has been paid to accuracy across human genetic variation. Here, we quantify the accuracy of DNA mixture analysis over 244 human groups. We find higher false inclusion rates for mixtures with more contributors, and for groups with lower genetic diversity. Even for two-contributor mixtures where one contributor is known and the reference group is correctly specified, false inclusion rates are 1e-5 or higher for 56 out of 244 groups. This means that, depending on multiple testing, some false inclusions may be expected. These false positives could be lessened with more selective and conservative use of DNA mixture analysis.

A Review of Probabilistic Genotyping Systems: EuroForMix, DNAStatistX and STRmix™

Probabilistic genotyping has become widespread. EuroForMix and DNAStatistX are both based upon maximum likelihood estimation using a γ model, whereas STRmix™ is a Bayesian approach that specifies prior distributions on the unknown model parameters. A general overview is provided of the historical development of probabilistic genotyping. Some general principles of interpretation are described, including: the application to investigative vs. evaluative reporting; detection of contamination events; inter and intra laboratory studies; numbers of contributors; proposition setting and validation of software and its performance. This is followed by details of the evolution, utility, practice and adoption of the software discussed.

A comparison of likelihood ratios obtained from EuroForMix and STRmix™

Likelihood ratios (LR) differences between the probabilistic genotyping software EuroForMix and STRmix™ are examined. After considering differences in the allele probabilities, the LRs from both software for an unambiguous single-source profile were identical (four significant figures). LRs from both software for an unambiguous single-source profile with alleles previously unseen in the allele frequency database (rare alleles) were the same (three significant figures) for θ = 0.01. Due to differences in the minimum allele frequencies, the LRs differed by three orders of magnitude when θ = 0. For both software, the LRs for a single-source dilution series decreased as the input amount decreased. The LRs from both software were within an order of magnitude for known contributors. The largest difference was where the target input amount was 0.0156 ng: The LR_EuroForMix was 2.1 × 10²⁵ and the LR_STRmix was 8.0 × 10²⁴ . Both software show similar LR behavior with respect to mixture ratio. For two person mixtures the LR increases for both the major and the minor as the ratio moves away from 1:1. The LR for the major stabilizes at about 3:1 whereas the LR for the minor reaches its maximum at about 3:1 and then declines. Greater differences in LR were observed between EuroForMix and STRmix™ for mixtures. One-hundred and twenty-nine mixtures from the PROVEDIt dataset were compared. LRs for 84% of the comparisons for known contributors without rare alleles were within two orders of magnitude. Five divergent results were investigated, and a manual intervention approach was applied where appropriate.

Examining performance and likelihood ratios for two likelihood ratio systems using the PROVEDIt dataset

A likelihood ratio (LR) system is defined as the entire pipeline of the measurement and interpretation processes where probabilistic genotyping software (PGS) is a piece of the whole LR system. To gain understanding on how two LR systems perform, a total of 154 two-person, 147 three-person, and 127 four-person mixture profiles of varying DNA quality, DNA quantity, and mixture ratios were obtained from the filtered (.CSV) files of the GlobalFiler 29 cycles 15s PROVEDIt dataset and deconvolved in two independently developed fully continuous programs, STRmix v2.6 and EuroForMix v2.1.0. Various parameters were set in each software and LR computations obtained from the two software were based on same/fixed EPG features, same pair of propositions, number of contributors, theta, and population allele frequencies. The ability of each LR system to discriminate between contributor (H1-true) and non-contributor (H2-true) scenarios was evaluated qualitatively and quantitatively. Differences in the numeric LR values and their corresponding verbal classifications between the two LR systems were compared. The magnitude of the differences in the assigned LRs and the potential explanations for the observed differences greater than or equal to 3 on the log10 scale were described. Cases of LR < 1 for H1-true tests and LR > 1 for H2-true tests were also discussed. Our intent is to demonstrate the value of using a publicly available ground truth known mixture dataset to assess discrimination performance of any LR system and show the steps used to understand similarities and differences between different LR systems. We share our observations with the forensic community and describe how examining more than one PGS with similar discrimination power can be beneficial, help analysts compare interpretation especially with low-template profiles or minor contributor cases, and be a potential additional diagnostic check even if software in use does contain certain diagnostic statistics as part of the output.

The a posteriori probability of the number of contributors when conditioned on an assumed contributor

Forensic DNA signal is notoriously challenging to assess, requiring computational tools to support its interpretation. Over-expressions of stutter, allele drop-out, allele drop-in, degradation, differential degradation, and the like, make forensic DNA profiles too complicated to evaluate by manual methods. In response, computational tools that make point estimates on the Number of Contributors (NOC) to a sample have been developed, as have Bayesian methods that evaluate an A Posteriori Probability (APP) distribution on the NOC. In cases where an overly narrow NOC range is assumed, the downstream strength of evidence may be incomplete insofar as the evidence is evaluated with an inadequate set of propositions. In the current paper, we extend previous work on NOCIt, a Bayesian method that determines an APP on the NOC given an electropherogram, by reporting on an implementation where the user can add assumed contributors. NOCIt is a continuous system that incorporates models of peak height (including degradation and differential degradation), forward and reverse stutter, noise, and allelic drop-out, while being cognizant of allele frequencies in a reference population. When conditioned on a known contributor, we found that the mode of the APP distribution can shift to one greater when compared with the circumstance where no known contributor is assumed, and that occurred most often when the assumed contributor was the minor constituent to the mixture. In a development of a result of Slooten and Caliebe (FSI:G, 2018) that, under suitable assumptions, establishes the NOC can be treated as a nuisance variable in the computation of a likelihood ratio between the prosecution and defense hypotheses, we show that this computation must not only use coincident models, but also coincident contextual information. The results reported here, therefore, illustrate the power of modern probabilistic systems to assess full weights-of-evidence, and to provide information on reasonable NOC ranges across multiple contexts.

A series of developmental validation tests for Number of Contributors platforms: Exemplars using NOCIt and a neural network

Complex DNA mixtures are challenging to interpret and require computational tools that aid in that interpretation. Recently, several computational methods that estimate the number of contributors (NOC) to a sample have been developed. Unlike analogous tools that interpret profiles and report LRs, NOC tools vary widely in their operational principle where some are Bayesian and others are machine learning tools. Conjunctionally, NOC tools may return a single n estimate, or a distribution on n. This vast array of constructs, coupled with a gap in standardized methods by which to validate NOC systems, warrants an exploration into the measures by which differing NOC systems might be tested for operations. In the current paper, we use two exemplar NOC systems: a probabilistic system named NOCIt, which renders an a posteriori probability (APP) distribution on the number of contributors given an electropherogram and an artificial neural network (ANN). NOCIt is a continuous Bayesian inference system incorporating models of peak height, degradation, differential degradation, forward and reverse stutter, noise and allelic drop-out while considering allele frequencies in a reference population. The ANN is also a continuous method, taking all the same features (barring degradation) into account. Unlike its Bayesian counterpart, it demands substantively more data to parameterize, requiring synthetic data. We explore each system's performance by conducting tests on 214 PROVEDIt mixtures where the limit of detection was 1-copy of DNA. We found that after a lengthy training period of approximately 24 h, the ANN's evaluation process was very fast and perfectly repeatable. In contrast, NOCIt only took a few minutes to train but took tens of minutes to complete each sample and was less repeatable. In addition, it rendered a probability distribution that was more sensitive and specific, affording a reasonable method by which to report all reasonable n that explain the evidence for a given sample. Whatever the method, by acknowledging the inherent differences between NOC systems, we demonstrate that validation constructs will necessarily be guided by the needs of the forensic domain and be dependent upon whether the laboratory seeks to assign a single n or range of n.

Application of a probabilistic genotyping software to MPS mixture STR data is supported by similar trends in LRs compared with CE data

The interpretation of short tandem repeat (STR) profiles can be challenging when, for example, alleles are masked due to allele sharing among contributors and/or when they are subject to drop-out, for instance from sample degradation. Mixture interpretation can be improved by increasing the number of STRs and/or loci with a higher discriminatory power. Both capillary electrophoresis (CE, 6-dye) and massively parallel sequencing (MPS) provide a platform for analysing relatively large numbers of autosomal STRs. In addition, MPS enables distinguishing between sequence variants, resulting in enlarged discriminatory power. Also, MPS allows for small amplicon sizes for all loci as spacing is not an issue, which is beneficial with degraded DNA. Altogether, MPS has the potential to increase the weights of evidence for true contributors to (complex) DNA profiles. In this study, likelihood ratio (LR) calculations were performed using STR profiles obtained with two different MPS systems and analysed using different settings: 1) MPS PowerSeq™ Auto System profiles analysed using FDSTools equipped with optimized settings such as noise correction, 2) ForenSeq™ DNA Signature Prep Kit profiles analysed using the default settings in the Universal Analysis Software (UAS), and 3) ForenSeq™ DNA Signature Prep Kit profiles analysed using FDSTools empirically adapted to cope with one-directional reads and provisional, basic settings. The LR calculations used genotyping data for two- to four-person mixtures varying for mixture proportion, level of drop-out and allele sharing and were generated with the continuous model EuroForMix. The LR results for the over 2000 sets of propositions were affected by the variation for the number of markers and analysis settings used in the three approaches. Nevertheless, trends for true and non-contributors, effects of replicates, assigned number of contributors, and model validation results were comparable for the three MPS approaches and alike the trends known for CE data. Based on this analogy, we regard the probabilistic interpretation of MPS STR data fit for forensic DNA casework. In addition, guidelines were derived on when to apply LR calculations to MPS autosomal STR data and report the corresponding results.

Interpol review of forensic biology and forensic DNA typing 2016-2019

This review paper covers the forensic-relevant literature in biological sciences from 2016 to 2019 as a part of the 19th Interpol International Forensic Science Managers Symposium. The review papers are also available at the Interpol website at: https://www.interpol.int/content/download/14458/file/Interpol%20Review%20Papers%202019.pdf.