The 'factor of two' issue in mixed DNA profiles

A collaborative study on the precision of the Markov chain Monte Carlo algorithms used for DNA profile interpretation

Several fully continuous probabilistic genotyping software (PGS) use Markov chain Monte Carlo algorithms (MCMC) to assign weights to different proposed genotype combinations at a locus. Replicate interpretations of the same profile in these software are expected not to produce identical weights and likelihood ratio (LR) values due to the Monte Carlo aspect. This paper reports a detailed precision study under reproducibility conditions conducted as a collaborative exercise across the National Institute of Standards and Technology (NIST), Federal Bureau of Investigation (FBI), and Institute of Environmental Science and Research (ESR). Replicate interpretations generated across the three laboratories used the same input files, software version, and settings but different random number seed and different computers. This work demonstrates that using different computers to analyze replicate interpretations does not contribute to any variations in LR values. The study quantifies the magnitude of differences in the assigned LRs that is only due to run-to-run MCMC variability and addresses the potential explanations for the observed differences.

Global survey on evaluative reporting on DNA evidence with regard to activity-level propositions

For many criminal cases, the source of who deposited the DNA is not what the prosecutor and the defense are trying to dispute. In court, the question may be how the DNA was deposited at the crime scene rather than who the DNA came from. Although laboratories in many countries have begun to evaluate DNA evidence given formal activity-level propositions (ALPs), it is unknown how much other forensic practitioners know and what they think about activity-level evaluative reporting (ALR). To collect this information, a survey with 21 questions was submitted to international forensic science organizations across Europe, Australia, South America, Canada, Asia, and Africa. The survey combined open-ended and multiple-choice questions and received 162 responses. Responses revealed a wide range of knowledge on the topic. Overall, most respondents were somewhat knowledgeable about ALR, ALP, and current practices in court and expressed their support of the concept. A majority of participants identified gaps and obstacles regarding ALR they would like to see addressed. Examples include (1) need for more education/training at all stakeholder levels, (2) need for more DNA evidence-related data under realistic case scenarios, (3) need to internally implement and validate a formalized and objective approach for reporting, and (4) in some countries the need to achieve court admissibility. This global survey gathered the current concerns of forensic DNA practitioners and outlined several operational concerns. The information can be used to advance the implementation of ALR in laboratories and court testimony worldwide.

A practical treatment of sensitivity analyses in activity level evaluations

Evaluations of forensic observations considering activity level propositions are becoming more common place in forensic institutions. A measure that can be taken to interrogate the evaluation for robustness is called sensitivity analysis. A sensitivity analysis explores the sensitivity of the evaluation to the data used when assigning probabilities, or to the level of uncertainty surrounding a probability assignment, or to the choice of various assumptions within the model. There have been a number of publications that describe sensitivity analysis in technical terms, and demonstrate their use, but limited literature on how that theory can be applied in practice. In this work we provide some simplified examples of how sensitivity analyses can be carried out, when they are likely to show that the evaluation is sensitive to underlying data, knowledge or assumptions, how to interpret the results of sensitivity analysis, and how the outcome can be reported. We also provide access to an application to conduct sensitivity analysis.

Addressing uncertain assumptions in DNA evidence evaluation

Evidential value of DNA mixtures is typically expressed by a likelihood ratio. However, selecting appropriate propositions can be contentious, because assumptions may need to be made around, for example, the contribution of a complainant's profile, or relatedness between contributors. A choice made one way or another disregards any uncertainty that may be present about such an assumption. To address this, a complex proposition that considers multiple sub-propositions with different assumptions may be more appropriate. While the use of complex propositions has been advocated in the literature, the uptake in casework has been limited. We provide a mathematical framework for evaluating DNA evidence given complex propositions and discuss its implementation in the DBLR™ software. The software simultaneously handles multiple mixed samples, reference profiles and relationships as described by a pedigree, which unlocks a variety of applications. We provide several examples to illustrate how complex propositions can efficiently evaluate DNA evidence. The addition of this feature to DBLR™ provides a tool to approach the long-accepted, but often impractical suggestion that propositions should be exhaustive within a case context.

Developmental validation of STRmix™ NGS, a probabilistic genotyping tool for the interpretation of autosomal STRs from forensic profiles generated using NGS

We describe the developmental validation of the probabilistic genotyping software - STRmix™ NGS - developed for the interpretation of forensic DNA profiles containing autosomal STRs generated using next generation sequencing (NGS) also known as massively parallel sequencing (MPS) technologies. Developmental validation was carried out in accordance with the Scientific Working Group on DNA Analysis Methods (SWGDAM) Guidelines for the Validation of Probabilistic Genotyping Systems and the International Society for Forensic Genetics (ISFG) recommendations and included sensitivity and specificity testing, accuracy, precision, and the interpretation of case-types samples. The results of developmental validation demonstrate the appropriateness of the software for the interpretation of profiles developed using NGS technology.

Internal Validation of MaSTR™ Probabilistic Genotyping Software for the Interpretation of 2–5 Person Mixed DNA Profiles

Mixed human deoxyribonucleic acid (DNA) samples present one of the most challenging pieces of evidence that a forensic analyst can encounter. When multiple contributors, stochastic amplification, and allele drop-out further complicate the mixture profile, interpretation by hand becomes unreliable and statistical analysis problematic. Probabilistic genotyping software has provided a tool to address complex mixture interpretation and provide likelihood ratios for defined sets of propositions. The MaSTR™ software is a fully continuous probabilistic system that considers a wide range of STR profile data to provide likelihood ratios on DNA mixtures. Mixtures with two to five contributors and a range of component ratios and allele peak heights were created to test the validity of MaSTR™ with data similar to real casework. Over 280 different mixed DNA profiles were used to perform more than 2600 analyses using different sets of propositions and numbers of contributors. The results of the analyses demonstrated that MaSTR™ provided accurate and precise statistical data on DNA mixtures with up to five contributors, including minor contributors with stochastic amplification effects. Tests for both Type I and Type II errors were performed. The findings in this study support that MaSTR™ is a robust tool that meets the current standards for probabilistic genotyping.

Hamiltonian Monte Carlo with strict convergence criteria reduces run-to-run variability in forensic DNA mixture deconvolution

MOTIVATION

Analysing mixed DNA profiles is a common task in forensic genetics. Due to the complexity of the data, such analysis is often performed using Markov Chain Monte Carlo (MCMC)-based genotyping algorithms. These trade off precision against execution time. When default settings (including default chain lengths) are used, as large as a 10-fold changes in inferred log-likelihood ratios (LR) are observed when the software is run twice on the same case. So far, this uncertainty has been attributed to the stochasticity of MCMC algorithms. Since LRs translate directly to strength of the evidence in a criminal trial, forensic laboratories desire LR with small run-to-run variability.

RESULTS

We present the use of a Hamiltonian Monte Carlo (HMC) algorithm that reduces run-to-run variability in forensic DNA mixture deconvolution by around an order of magnitude without increased runtime. We achieve this by enforcing strict convergence criteria. We show that the choice of convergence metric strongly influences precision. We validate our method by reproducing previously published results for benchmark DNA mixtures (MIX05, MIX13, and ProvedIt). We also present a complete software implementation of our algorithm that is able to leverage GPU acceleration for the inference process. In the benchmark mixtures, on consumer-grade hardware, the runtime is less than 7 min for 3 contributors, less than 35 min for 4 contributors, and less than an hour for 5 contributors with one known contributor.

A Logical Framework for Forensic DNA Interpretation

The forensic community has devoted much effort over the last decades to the development of a logical framework for forensic interpretation, which is essential for the safe administration of justice. We review the research and guidelines that have been published and provide examples of how to implement them in casework. After a discussion on uncertainty in the criminal trial and the roles that the DNA scientist may take, we present the principles of interpretation for evaluative reporting. We show how their application helps to avoid a common fallacy and present strategies that DNA scientists can apply so that they do not transpose the conditional. We then discuss the hierarchy of propositions and explain why it is considered a fundamental concept for the evaluation of biological results and the differences between assessing results given propositions that are at the source level or the activity level. We show the importance of pre-assessment, especially when the questions relate to the alleged activities, and when transfer and persistence need to be considered by the scientists to guide the court. We conclude with a discussion on statement writing and testimony. This provides guidance on how DNA scientists can report in a balanced, transparent, and logical way.

A mixed DNA profile controversy revisited

Semaan et al. (J Forensic Res, 2020, 11, 453) discuss a mock case "where eight different individuals [P₁ through P₈ ] could not be excluded in a mixed DNA analysis. Even though … expert DNA mixture analysis software was used." Two of these are the true donors. The LRs reported are incorrect due to the incorrect entry of propositions into LRmix Studio. This forced the software to account for most of the alleles as drop-in, resulting in LRs 60-70 orders of magnitude larger than expected. P₁ , P₂ , P₄ , P₅ , and P₈ can be manually excluded using peak heights. This has relevance when using LRmix which does not use peak heights. We extend the work using the same two reference genotypes who were the true contributors as Semaan et al. (J Forensic Res, 2020, 11, 453). We simulate three two-donor mixtures with peak heights using these two genotypes and analyze using STRmix™. For the simulated 1:1 mixture, one of the non-donors' LRs supported him being a contributor when no conditioning was used. When considered in combination with any other potential donors (i.e., with conditioning), this non-donor was correctly eliminated. For the 3:1 mixture, all results correctly supported that the non-donors were not contributors. The low-template 4:1 mixture LRs with no conditioning showed support for all eight profiles as donors. However, the results from pair-wise conditioning showed that only the two ground truth donors had LRs supporting that they were contributors to the mixture. We recommend the use of peak heights and conditioning profiles, as this allows better sensitivity and specificity even when the persons share many alleles.

A Review of Probabilistic Genotyping Systems: EuroForMix, DNAStatistX and STRmix™

Probabilistic genotyping has become widespread. EuroForMix and DNAStatistX are both based upon maximum likelihood estimation using a γ model, whereas STRmix™ is a Bayesian approach that specifies prior distributions on the unknown model parameters. A general overview is provided of the historical development of probabilistic genotyping. Some general principles of interpretation are described, including: the application to investigative vs. evaluative reporting; detection of contamination events; inter and intra laboratory studies; numbers of contributors; proposition setting and validation of software and its performance. This is followed by details of the evolution, utility, practice and adoption of the software discussed.

Searching CODIS with binary conversions of STRmix interpretations

The predominant approach to interpreting autosomal STR DNA typing results is through the use of probabilistic genotyping software. The primary output from such software is a list of genotypes and associated genotype weights representing the likelihood of observing the typing results if an individual or set of individuals with that genotype(s) was the donor of the evidence DNA. While such lists are used by probabilistic software to calculate likelihood ratios based upon a pair of propositions regarding specific donors of the DNA, they are not directly amendable to entry into law enforcement databases that use the Combined DNA Index System (CODIS) software. An approach to creating CODIS-compatible profiles from the output of the probabilistic genotyping program STRmix is discussed and assessed for sensitivity and specificity. Combining this information with an assessment of the weight-of-evidence potential contained in the STRmix interpretation, pragmatic guidance was developed regarding the creation and searching of CODIS profiles in a way that balances the detection of investigative leads with the potential burdensome effects on workload.

Secondary DNA transfer by working gloves

With the development of highly sensitive STR profiling methods, combined with sound statistical tools, DNA analysis on the (sub-)source level is hardly ever seriously questioned in court. More often, the exact mode of DNA transfer to the crime scene is questioned. In burglary cases, in particular when gloves are worn, secondary DNA transfer is often discussed as explanation for finding a DNA profile matching the accused because it is well known that gloves can act as a potential vector for indirect DNA transfer. In this study we investigated the shedder status as a possible factor influencing the extent of secondary DNA transfer to a crime scene, with the person committing the crime wearing working gloves. Firstly, the shedder status for 40 participants (20 male, 20 female) was determined, following a previously published procedure. Good shedders (n = 12) were found to deposit a higher amount and quality of DNA onto objects, compared to bad shedders (n = 25). Secondly, participants were paired into four groups (good with good; good with bad; bad with good; bad with bad), each group consisting of five pairs. The first participant (P1) of each pair used working gloves to pack and carry a box to simulate a house move. Two days later, the second participant (P2) of the pair wore the same pair of gloves to simulate a burglary, using a screwdriver as a break-in tool. After taking swabs of the outside and inside of a glove (primary DNA transfer) and the handle of the screwdriver (secondary DNA transfer), full DNA analysis was performed. Our experiments show that good shedders, overall, deposit more DNA than bad shedders, both onto the outside and the inside of the glove, regardless of being P1 or P2. When conducting the experiments with two participants sharing the same shedder status, no significant differences occurred in the number of deposited alleles. In six out of 19 cases a DNA profile matching P1 was found (binary LR>10⁶) on the screwdriver and in all six cases P1 was a good shedder. Our results indicate that the shedder status of an individual affects the extent of DNA transfer. They further confirm the possibility of an innocent person's DNA profile being found on an object they never handled.

The Probabilistic Genotyping Software STRmix: Utility and Evidence for its Validity

Forensic DNA interpretation is transitioning from manual interpretation based usually on binary decision-making toward computer-based systems that model the probability of the profile given different explanations for it, termed probabilistic genotyping (PG). Decision-making by laboratories to implement probability-based interpretation should be based on scientific principles for validity and information that supports its utility, such as criteria to support admissibility. The principles behind STRmix™ are outlined in this study and include standard mathematics and modeling of peak heights and variability in those heights. All PG methods generate a likelihood ratio (LR) and require the formulation of propositions. Principles underpinning formulations of propositions include the identification of reasonably assumed contributors. Substantial data have been produced that support precision, error rate, and reliability of PG, and in particular, STRmix™. A current issue is access to the code and quality processes used while coding. There are substantial data that describe the performance, strengths, and limitations of STRmix™, one of the available PG software.

DNA commission of the International society for forensic genetics: Assessing the value of forensic biological evidence - Guidelines highlighting the importance of propositions: Part I: evaluation of DNA profiling comparisons given (sub-) source propositions

The interpretation of evidence continues to be one of the biggest challenges facing the forensic community. This is the first of two papers intended to provide advice on difficult aspects of evaluation and in particular on the formulation of propositions. The scientist has a dual role: investigator (crime-focused), where often there is no suspect available and a database search may be required; evaluator (suspect-focused), where the strength of evidence is assessed in the context of the case. In investigative mode, generally the aim is to produce leads regarding the source of the DNA. Sub-source level propositions will be adequate to help identify potential suspects who can be further investigated by the authorities. Once in evaluative mode, given the defence version of events of the person of interest, it may become necessary to consider alternatives that go beyond the source of the DNA (i.e., to consider activity level propositions). In the evaluation phase, it is crucial that formulation of propositions allows the assessment of all the results that will help with the issue at hand. Propositions should therefore be precise (indication of the number of contributors, information on the relevant population etc.), be about causes, not effects (e.g. a 'matching' DNA profile) and to avoid bias, must not be findings-led. This means that ideally, propositions should be decided based on the case information and before the results of the comparisons are known. This paper primarily reflects upon what has been coined as "sub-source level propositions". These are restricted to the evaluation of the DNA profiles themselves, and help answer the issue regarding the source of the DNA. It is to be emphasised that likelihood ratios given sub-source level propositions cannot be carried over to a different level - for example, activity level propositions, where the DNA evidence is put into the context of the alleged activities. This would be highly misleading and could give rise to miscarriages of justice; this will be discussed in a second paper. The value of forensic results depends not only on propositions, but also on the type of results (e.g. allelic designations, peak heights, replicates) and upon the model used: it is therefore important to discuss these aspects. Finally, since communication is key to help understanding by courts, we will explore how to convey the value of the results and explain the importance of avoiding the practice of transposing the conditional.

Evaluating forensic biology results given source level propositions

The evaluation of forensic evidence can occur at any level within the hierarchy of propositions depending on the question being asked and the amount and type of information that is taken into account within the evaluation. Commonly DNA evidence is reported given propositions that deal with the sub-source level in the hierarchy, which deals only with the possibility that a nominated individual is a source of DNA in a trace (or contributor to the DNA in the case of a mixed DNA trace). We explore the use of information obtained from examinations, presumptive and discriminating tests for body fluids, DNA concentrations and some case circumstances within a Bayesian network in order to provide assistance to the Courts that have to consider propositions at source level. We use a scenario in which the presence of blood is of interest as an exemplar and consider how DNA profiling results and the potential for laboratory error can be taken into account. We finish with examples of how the results of these reports could be presented in court using either numerical values or verbal descriptions of the results.

Testing likelihood ratios produced from complex DNA profiles

The performance of any model used to analyse DNA profile evidence should be tested using simulation, large scale validation studies based on ground-truth cases, or alignment with trends predicted by theory. We investigate a number of diagnostics to assess the performance of the model using Hd true tests. Of particular focus in this work is the proportion of comparisons to non-contributors that yield a likelihood ratio (LR) higher than or equal to the likelihood ratio of a known contributor (LRPOI), designated as p, and the average LR for Hd true tests. Theory predicts that p should always be less than or equal to 1/LRPOI and hence the observation of this in any particular case is of limited use. A better diagnostic is the average LR for Hd true which should be near to 1. We test the performance of a continuous interpretation model on nine DNA profiles of varying quality and complexity and verify the theoretical expectations.