Examining performance and likelihood ratios for two likelihood ratio systems using the PROVEDIt dataset

A collaborative study on the precision of the Markov chain Monte Carlo algorithms used for DNA profile interpretation

Several fully continuous probabilistic genotyping software (PGS) use Markov chain Monte Carlo algorithms (MCMC) to assign weights to different proposed genotype combinations at a locus. Replicate interpretations of the same profile in these software are expected not to produce identical weights and likelihood ratio (LR) values due to the Monte Carlo aspect. This paper reports a detailed precision study under reproducibility conditions conducted as a collaborative exercise across the National Institute of Standards and Technology (NIST), Federal Bureau of Investigation (FBI), and Institute of Environmental Science and Research (ESR). Replicate interpretations generated across the three laboratories used the same input files, software version, and settings but different random number seed and different computers. This work demonstrates that using different computers to analyze replicate interpretations does not contribute to any variations in LR values. The study quantifies the magnitude of differences in the assigned LRs that is only due to run-to-run MCMC variability and addresses the potential explanations for the observed differences.

An overview of log likelihood ratio cost in forensic science - Where is it used and what values can we expect?

There is increasing support for reporting evidential strength as a likelihood ratio (LR) and increasing interest in (semi-)automated LR systems. The log-likelihood ratio cost (Cllr) is a popular metric for such systems, penalizing misleading LRs further from 1 more. Cllr = 0 indicates perfection while Cllr = 1 indicates an uninformative system. However, beyond this, what constitutes a "good" Cllr is unclear. Aiming to provide handles on when a Cllr is "good", we studied 136 publications on (semi-)automated LR systems. Results show Cllr use heavily depends on the field, e.g., being absent in DNA analysis. Despite more publications on automated LR systems over time, the proportion reporting Cllr remains stable. Noticeably, Cllr values lack clear patterns and depend on the area, analysis and dataset. As LR systems become more prevalent, comparing them becomes crucial. This is hampered by different studies using different datasets. We advocate using public benchmark datasets to advance the field.

Single-cell investigative genetics: Single-cell data produces genotype distributions concentrated at the true genotype across all mixture complexities

In the absence of a suspect the forensic aim is investigative, and the focus is one of discerning what genotypes best explain the evidence. In traditional systems, the list of candidate genotypes may become vast if the sample contains DNA from many donors or the information from a minor contributor is swamped by that of major contributors, leading to lower evidential value for a true donor's contribution and, as a result, possibly overlooked or inefficient investigative leads. Recent developments in single-cell analysis offer a way forward, by producing data capable of discriminating genotypes. This is accomplished by first clustering single-cell data by similarity without reference to a known genotype. With good clustering it is reasonable to assume that the scEPGs in a cluster are of a single contributor. With that assumption we determine the probability of a cluster's content given each possible genotype at each locus, which is then used to determine the posterior probability mass distribution for all genotypes by application of Bayes' rule. A decision criterion is then applied such that the sum of the ranked probabilities of all genotypes falling in the set is at least 1-α. This is the credible genotype set and is used to inform database search criteria. Within this work we demonstrate the salience of single-cell analysis by performance testing a set of 630 previously constructed admixtures containing up to 5 donors of balanced and unbalanced contributions. We use scEPGs that were generated by isolating single cells, employing a direct-to-PCR extraction treatment, amplifying STRs that are compliant with existing national databases and applying post-PCR treatments that elicit a detection limit of one DNA copy. We determined that, for these test data, 99.3% of the true genotypes are included in the 99.8% credible set, regardless of the number of donors that comprised the mixture. We also determined that the most probable genotype was the true genotype for 97% of the loci when the number of cells in a cluster was at least two. Since efficient investigative leads will be borne by posterior mass distributions that are narrow and concentrated at the true genotype, we report that, for this test set, 47,900 (86%) loci returned only one credible genotype and of these 47,551 (99%) were the true genotype. When determining the LR for true contributors, 91% of the clusters rendered LR>1018, showing the potential of single-cell data to positively affect investigative reporting.

QHFSS DNA laboratory - ISO/IEC 17025 conformance and accreditation

This paper reviews evidence placed before a Commission of Inquiry (CoI) established by the State of Queensland, Australia, to consider the quality and reliability of DNA evidence. It also assesses whether the criticism levied in that report, of ISO/IEC 17025 being insufficient to assure the quality and reliability of DNA evidence, is warranted. The main conclusion drawn is that properly applied and embraced as a means of continuous improvement, conformance with ISO/IEC 17025:2017 alone is sufficient to assure the quality and reliability of the scientific outputs from a forensic science laboratory. Furthermore, it is clear from the observations and findings of the CoI and those recorded in this paper that the forensic science laboratory in question did not conform to ISO/IEC 17025:2017. Had it done so then the risk of the quality failures that led to the CoI would at least have been reduced and perhaps even avoided.

A diagnosis of the primary difference between EuroForMix and STRmix™

There is interest in comparing the output, principally the likelihood ratio, from the two probabilistic genotyping software EuroForMix (EFM) and STRmix™. Many of these comparison studies are descriptive and make little or no effort to diagnose the cause of difference. There are fundamental differences between EFM and STRmix™ that are causative of the largest set of likelihood ratio differences. This set of differences is for false donors where there are many instances of LRs just above or below 1 for EFM that give much lower LRs in STRmix™. This is caused by the separate estimation of parameters such as allele height variance and mixture proportion using MLE under Hp and Ha for EFM. This can result in very different estimations of these parameters under Hp and Ha . It results in a departure from calibration for EFM in the region of LRs just above and below 1.

Variational inference accelerates accurate DNA mixture deconvolution

We investigate a class of DNA mixture deconvolution algorithms based on variational inference, and we show that this can significantly reduce computational runtimes with little or no effect on the accuracy and precision of the result. In particular, we consider Stein Variational Gradient Descent (SVGD) and Variational Inference (VI) with an evidence lower-bound objective. Both provide alternatives to the commonly used Markov-Chain Monte-Carlo methods for estimating the model posterior in Bayesian probabilistic genotyping. We demonstrate that both SVGD and VI significantly reduce computational costs over the current state of the art. Importantly, VI does so without sacrificing precision or accuracy, presenting an overall improvement over previously published methods.

Magnetic bead-based separation of sperm cells from semen-vaginal fluid mixed stains using an anti-ACRBP antibody

Forensic DNA analysis of semen-vaginal fluid mixed stains is essential and necessary in sexual assault cases. Here, we used a magnetic bead conjugated acrosin binding protein (ACRBP) antibody to separate and enrich sperm cells from mixed stains. Previously, western blotting indicated that ACRBP was specifically expressed in sperm cells, but not in female blood and epithelial cells, while immunofluorescence data showed ACRBP was localized to the acrosome in sperm cells. In our study, sperm were separated from mixed samples at three sperm cell/female buccal epithelial cell ratios (10³:10³; 10³:10⁴; and 10³:10⁵) using a magnetic bead conjugated ACRBP antibody. Subsequently, 23 autosomal short tandem repeat (STR) loci were amplified using the Huaxia™ Platinum PCR Amplification System and genotyped using capillary electrophoresis. The genotyping success rate for STR loci was 90% when the sperm to female buccal epithelial cell ratio was > 1:100 in mixed samples. Our results suggest that the magnetic bead conjugated ACRBP antibody is effective for isolating sperm cells in sexual assault cases.

Uncertainty in probabilistic genotyping of low template DNA: A case study comparing STRMix™ and TrueAllele™

Two probabilistic genotyping (PG) programs, STRMix™ and TrueAllele™, were used to assess the strength of the same item of DNA evidence in a federal criminal case, with strikingly different results. For STRMix, the reported likelihood ratio in favor of the non-contributor hypothesis was 24; for TrueAllele it ranged from 1.2 million to 16.7 million, depending on the reference population. This case report seeks to explain why the two programs produced different results and to consider what the difference tells us about the reliability and trustworthiness of these programs. It uses a locus-by-locus breakdown to trace the differing results to subtle differences in modeling parameters and methods, analytic thresholds, and mixture ratios, as well as TrueAllele's use of an ad hoc procedure for assigning LRs at some loci. These findings illustrate the extent to which PG analysis rests on a lattice of contestable assumptions, highlighting the importance of rigorous validation of PG programs using known-source test samples that closely replicate the characteristics of evidentiary samples. The article also points out misleading aspects of the way STRMix and TrueAllele results are routinely presented in reports and testimony and calls for clarification of forensic reporting standards to address those problems.

Analysis of the Hamiltonian Monte Carlo genotyping algorithm on PROVEDIt mixtures including a novel precision benchmark

We provide an internal validation study of a recently published precise DNA mixture algorithm based on Hamiltonian Monte Carlo sampling (Susik et al., 2022). We provide results for all 428 mixtures analysed by Riman et al. (2021) and compare the results with two state-of-the-art software products: STRmix™  v2.6 and Euroformix v3.4.0. The comparison shows that the Hamiltonian Monte Carlo method provides reliable values of likelihood ratios (LRs) close to the other methods. We further propose a novel large-scale precision benchmark and quantify the precision of the Hamiltonian Monte Carlo method, indicating its improvements over existing solutions. Finally, we analyse the influence of the factors discussed by Buckleton et al. (2022).

Recent advances in forensic biology and forensic DNA typing: INTERPOL review 2019-2022

This review paper covers the forensic-relevant literature in biological sciences from 2019 to 2022 as a part of the 20th INTERPOL International Forensic Science Managers Symposium. Topics reviewed include rapid DNA testing, using law enforcement DNA databases plus investigative genetic genealogy DNA databases along with privacy/ethical issues, forensic biology and body fluid identification, DNA extraction and typing methods, mixture interpretation involving probabilistic genotyping software (PGS), DNA transfer and activity-level evaluations, next-generation sequencing (NGS), DNA phenotyping, lineage markers (Y-chromosome, mitochondrial DNA, X-chromosome), new markers and approaches (microhaplotypes, proteomics, and microbial DNA), kinship analysis and human identification with disaster victim identification (DVI), and non-human DNA testing including wildlife forensics. Available books and review articles are summarized as well as 70 guidance documents to assist in quality control that were published in the past three years by various groups within the United States and around the world.

Gemeinsame Empfehlungen der Projektgruppe „Biostatistische DNA-Berechnungen“ und der Spurenkommission zur biostatistischen Bewertung forensischer DNA-analytischer Befunde mit vollkontinuierlichen Modellen (VKM)

Die biostatistische Bewertung DNA-analytischer Befunde unterstützt Gerichte bei der Einschätzung des Beweiswertes hinsichtlich einer möglichen Spurenbeteiligung durch eine zu betrachtende Person (engl. „Person Of Interest“; POI). Um die Vergleichbarkeit derartiger Berechnungen auf Grundlage etablierter wissenschaftlicher Standards zu gewährleisten, wurden bereits in der Vergangenheit entsprechende Empfehlungen im nationalen Konsens formuliert.

Mit Einführung sog. vollkontinuierlicher Modelle (VKM) für die probabilistische Genotypisierung, die u. a. die Signalintensitäten eines Elektropherogramms berücksichtigen, wurde eine Ergänzung zu den damaligen Empfehlungen erforderlich. VKM erlauben eine biostatistische Bewertung von Spuren mit möglichen Drop-in- und Drop-out-Ereignissen und wahrscheinlichkeitsbasierte Prognosen der zu einer Mischspur beitragenden Genotypen („Deconvolution“).

Die vorliegende Veröffentlichung enthält Empfehlungen zum Einsatz VKM-basierter Software und zur Berichterstattung vollkontinuierlicher LR-Werte (engl. „Fully Continuous Likelihood Ratios“; LRfc). Sie empfiehlt bei schwierig zu interpretierenden Befunden eine VKM-Berechnung zur Bewertung einer Spurenlegerschaft. Die VKM-Berechnung ersetzt die bisher in Ausnahmefällen als hinnehmbar erachtete Vorgehensweise einer binären Berechnung unter Ausklammern einzelner Merkmalssysteme. Der Einsatz von VKM erfordert eine umfassende Anwenderschulung sowie eine Validierung und Verifizierung gemäß den Vorgaben der Programmanbieter. Mit der Empfehlung von LRfc-Schwellenwerten soll eine sichere, vergleichbare Anwendung von VKM gewährleistet werden.

Hamiltonian Monte Carlo with strict convergence criteria reduces run-to-run variability in forensic DNA mixture deconvolution

MOTIVATION

Analysing mixed DNA profiles is a common task in forensic genetics. Due to the complexity of the data, such analysis is often performed using Markov Chain Monte Carlo (MCMC)-based genotyping algorithms. These trade off precision against execution time. When default settings (including default chain lengths) are used, as large as a 10-fold changes in inferred log-likelihood ratios (LR) are observed when the software is run twice on the same case. So far, this uncertainty has been attributed to the stochasticity of MCMC algorithms. Since LRs translate directly to strength of the evidence in a criminal trial, forensic laboratories desire LR with small run-to-run variability.

RESULTS

We present the use of a Hamiltonian Monte Carlo (HMC) algorithm that reduces run-to-run variability in forensic DNA mixture deconvolution by around an order of magnitude without increased runtime. We achieve this by enforcing strict convergence criteria. We show that the choice of convergence metric strongly influences precision. We validate our method by reproducing previously published results for benchmark DNA mixtures (MIX05, MIX13, and ProvedIt). We also present a complete software implementation of our algorithm that is able to leverage GPU acceleration for the inference process. In the benchmark mixtures, on consumer-grade hardware, the runtime is less than 7 min for 3 contributors, less than 35 min for 4 contributors, and less than an hour for 5 contributors with one known contributor.

High-quality data from a forensically relevant single-cell pipeline enabled by low PBS and proteinase K concentrations

Interpreting forensic DNA signal is arduous since the total intensity is a cacophony of signal from noise, artifact, and allele from an unknown number of contributors (NOC). An alternate to traditional bulk-processing pipelines is a single-cell one, where the sample is collected, and each cell is sequestered resulting in n single-source, single-cell EPGs (scEPG) that must be interpreted using applicable strategies. As with all forensic DNA interpretation strategies, high quality electropherograms are required; thus, to enhance the credibility of single-cell forensics, it is necessary to produce an efficient direct-to-PCR treatment that is compatible with prevailing downstream laboratory processes. We incorporated the semi-automated micro-fluidic DEPArray™ technology into the single-cell laboratory and optimized its implementation by testing the effects of four laboratory treatments on single-cell profiles. We focused on testing effects of phosphate buffer saline (PBS) since it is an important reagent that mitigates cell rupture but is also a PCR inhibitor. Specifically, we explored the effect of decreasing PBS concentrations on five electropherogram-quality metrics from 241 leukocytes: profile drop-out, allele drop-out, allele peak heights, peak height ratios, and scEPG sloping. In an effort to improve reagent use, we also assessed two concentrations of proteinase K. The results indicate that decreasing PBS concentrations to 0.5X or 0.25X improves scEPG quality, while modest modifications to proteinase K concentrations did not significantly impact it. We, therefore, conclude that a lower than recommended proteinase K concentration coupled with a lower than recommended PBS concentration results in enhanced scEPGs within the semi-automated single-cell pipeline.