Biasability and reliability of expert forensic document examiners

Contextual bias in forensic toxicology decisions: A follow-up empirical study from China

The impact of contextual bias has been demonstrated repeatedly across forensic domains; however, research on this topic in forensic toxicology is very limited. In our previous study, experimental data from only one context version were compared with the actual forensic biasing casework. As a follow-up, this controlled experiment with 159 forensic toxicology practitioners was conducted, to test whether knowledge of different contextual information influenced their forensic decision-making. Participants in different context groups were tasked to identify testing strategies for carbon monoxide and opiate drugs. The results of chi-squared tests for their selections and two context groups exhibited statistically significant differences (p < 0.05 or p < 0.01). These findings show contextual information can bias forensic toxicology decisions about testing strategies, despite it is a relatively objective domain in forensic science.

Contextual bias by Forensic Document Examination trainees: An empirical study from China

The impact of contextual bias has been repeatedly demonstrated across forensic domains; however, research on this topic in China is scarce. To examine the prevalence of contextual bias in pattern feature-comparison disciplines, we conducted an experiment involving 24 forensic document examination students. The aim was to determine whether knowledge of different contextual information influenced their forensic decision-making. Participants were divided into different context groups and tasked with examining whether questioned signatures with ambiguous features matched reference signatures. The results of independent-samples t-tests for their decision score data in the two context groups exhibited a statistically significant difference (p < 0.05, Cohen's d > 0.8). Moreover, the submitted forensic reports by participants disclosed a biased evaluation of handwriting features. These findings show how contextual information can bias forensic decision-making in handwriting examination. Context management with complementary strategies such as case triage, cognitive training and decision-making transparency must be implemented to minimize bias in handwriting examination.

The effects of cognitive bias, examiner expertise, and stimulus material on forensic evidence analysis

Forensic examiners have come under scrutiny due to high-profile exonerations, highlighting the consequences that contextual bias can have on investigations. Researchers have proposed solutions to reduce the effects of bias including blind testing and redacting task-irrelevant information. Practitioners have concerns over the limitations of some of this research that uses untrained students to examine complex pieces of forensic evidence (e.g., fingerprints) (1; but see 2 for studies including trained experts and/or actual casework). This study sought to (a) examine the effect of contextual bias on examiners' evaluation of forensic evidence by varying the amount of pre-comparison information available to participants, (b) compare student and expert examiners' performance and their vulnerability to contextual bias, and (c) examine the effects of contextual bias on examiners' evaluation of different types of forensic evidence. Expert fingerprint examiners and student participants were presented with varying amounts of pre-comparison case information and compared matching and non-matching fingerprint and footwear impression evidence. Results suggest no effects of blinding examiners from case information or redacting task-irrelevant information. As expected, expert fingerprint examiners were more likely to correctly identify matching fingerprints and correctly exclude non-matching fingerprints than students. However, expert fingerprint examiners were no better than student participants at comparing footwear impression evidence. These findings suggest that sample, stimulus selection, and discipline-specific training matter when investigating bias in forensic decision making. These findings suggest caution when using forensic stimuli with student samples to investigate forensic decision-making and highlight the need for more research on redaction procedures.

The most consistent finding in forensic science is inconsistency

The most consistent finding in many forensic science domains is inconsistency (i.e., lack of reliability, reproducibility, repeatability, and replicability). The lack of consistency is a major problem, both from a scientific and a criminal justice point of view. Examining forensic conclusion data, from across many forensic domains, highlights the underlying cognitive issues and offers a better understanding of the issues and challenges. Such insights enable the development of ways to minimize these inconsistencies and move forward. The aim is to highlight the problem, so that it can be minimized and the reliability of forensic science evidence can be improved.

The scientific reinvention of forensic science

Forensic science is undergoing an evolution in which a long-standing "trust the examiner" focus is being replaced by a "trust the scientific method" focus. This shift, which is in progress and still partial, is critical to ensure that the legal system uses forensic information in an accurate and valid way. In this Perspective, we discuss the ways in which the move to a more empirically grounded scientific culture for the forensic sciences impacts testing, error rate analyses, procedural safeguards, and the reporting of forensic results. However, we caution that the ultimate success of this scientific reinvention likely depends on whether the courts begin to engage with forensic science claims in a more rigorous way.

Challenges to reasoning in forensic science decisions

The success of forensic science depends heavily on human reasoning abilities. Although we typically navigate our lives well using those abilities, decades of psychological science research shows that human reasoning is not always rational. In addition, forensic science often demands that its practitioners reason in non-natural ways. This article addresses how characteristics of human reasoning (either specific to an individual or in general) and characteristics of situations (either specific to a case or in general in a lab) can contribute to errors before, during, or after forensic analyses. In feature comparison judgments, such as fingerprints or firearms, a main challenge is to avoid biases from extraneous knowledge or arising from the comparison method itself. In causal and process judgments, for example fire scenes or pathology, a main challenge is to keep multiple potential hypotheses open as the investigation continues. Considering the contributions to forensic science judgments by persons, situations, and their interaction, reveals ways to develop procedures to decrease errors and improve accuracy.

Metric forensic anthropology decisions: Reliability and biasability of sectioning-point-based sex estimates

Subjective decisions make human cognitive processes more susceptible to bias and error. Specifically, research indicates that additional context biases forensic anthropologists' morphological analyses. To address whether metric analyses are also subject to bias, we conducted a pilot study in which 52 experienced osteologists measured a difficult-to-classify human femur, with or without additional contextual information. Using a metric sectioning-point sex-estimation method, participants provided a sex estimate for individual skeletal element(s) and, when given multiple elements, the combined skeletal assemblage. Control group participants (n = 24) measured only the femur. In addition to the femur, bias group participants (n = 28) either measured a female humerus and viewed a female-biasing photograph (n = 14) or measured a male humerus and viewed a male-biasing photograph (n = 14). We explored whether the experts in the different groups would differ in: (1) femoral measurements; (2) femoral sex-estimation conclusions; and (3) final sex-estimation conclusions for the skeletal assemblage. Although the femoral measurements and femoral sex estimates were comparable across groups, the overall sex estimates in the female-biased group were impacted by contextual information-differing from both the control and male-biased groups (p < 0.001). Our results demonstrate that cognitive bias can occur even in metric sex-estimation conclusions. Specifically, this occurred when the metric data and single-element sex estimates were synthesized into an overall estimate. Thus, our results suggest that metric methods are most vulnerable to bias when data are synthesized into an overall conclusion, highlighting the need for bias countermeasures and comprehensive statistical frameworks for synthesizing metric data to mitigate the effects of cognitive bias.

A hierarchy of expert performance as applied to forensic anthropology

Due to their medicolegal repercussions, forensic anthropology conclusions must be reliable, consistent, and minimally compromised by bias. Yet, a synthetic analysis of the reliability and biasability of the discipline's methods has not yet been conducted. To do so, this study utilized Dror's (2016) hierarchy of expert performance (HEP), an eight-level model aimed at examining intra- and inter-expert reliability and biasability (the potential for cognitive bias) within the literature of forensic science disciplines. A systematic review of the forensic anthropology literature was conducted (1972-present), including papers published in the American Journal of Physical Anthropology, Forensic Anthropology, Forensic Science International, and the Journal of Forensic Sciences and Anthropology Section abstracts published in the Proceedings of the Annual Meeting of the AAFS which matched keywords such as "forensic anthropology," "bias," "reliability," "cognition," "cognitive," or "error." The resulting forensic anthropology HEP showcases areas that have ample research and areas where more research can be conducted. Specifically, statistically significant increases in reliability (p < 0.001) and biasability (p < 0.001) publications were found since 2009 (publication of the NAS report). Extensive research examined the reliability of forensic anthropological observations and conclusions (n = 744 publications). However, minimal research investigated the biasability of forensic anthropological observations and conclusions (n = 20 publications). Notably, while several studies demonstrated the biasing effect of extraneous information on anthropological morphological assessments, there was no research into these effects on anthropological metric assessments. The findings revealed by the forensic anthropology HEP can help to guide future research, ultimately informing the development and refinement of best-practice standards for the discipline.