Who can judge the accuracy of eyewitness statements? A comparison of professionals and lay-persons

Comparing human evaluations of eyewitness statements to a machine learning classifier under pristine and suboptimal lineup administration procedures

Recent work highlights the ability of verbal machine learning classifiers to distinguish between accurate and inaccurate recognition memory decisions (Dobbins, 2022; Dobbins & Kantner, 2019; Seale-Carlisle, Grabman, & Dodson, 2022). Given the surge of interest in these modeling techniques, there is an urgent need to investigate verbal classifiers' limitations - particularly in applied contexts such as when police collect eyewitness's confidence statements. We find that confirmatory feedback (e.g., "This study now has a total of 87 participants, 84 of them made the same decision as you!") weakens the relationship between identification accuracy and verbal classifier scores to a similar degree as mock witnesses' numeric confidence judgments (Experiment 1). Crucially, for the first time, we compare the discriminative value of verbal classifier scores to the ratings of human evaluators who assessed the identical verbal confidence statements (Experiment 2). Our results suggest that human evaluators outperform the classifier when mock witnesses received no feedback; however, the classifier matches (or exceeds) the performance of human evaluators when mock witnesses received confirmatory feedback. Providing lineup information to human evaluators resulted in a worse ability to distinguish between correct and filler identifications, suggesting that this particular information may encourage the use of inappropriate heuristics when rendering accuracy judgments. Overall, these results suggest that the utility of verbal classifiers may be enhanced when contextual effects (e.g., lineup presence) impair human estimates of others' performance, but that translating witnesses' statements into classifier scores will not fix the problems of an improperly conducted lineup procedure.

Discernment of Children's True and False Memory Reports: Police Officers and Laypersons

Adults' ability to accurately evaluate children's statements can have far-reaching consequences within the legal system. This study examined the evaluations of police officers ("experts") and laypersons ("nonexperts") when presented with videotaped interviews of children aged 3 and 5 years who provided either true or false reports or denials. Participants were drawn from several counties in the eastern United States. Children's interview statements fell within four statement types: accurate reports, false reports, accurate denials, and false denials. Both groups of participants displayed overbelief in false denials. Several control variables predicted accuracy, including children's age and children's race. A significant interaction emerged: Experts (vs. nonexperts) had greater odds of being accurate when judging false reports (vs. false denials). These findings highlight the challenges adults face when distinguishing between various types of children's statements. The results have important implications for legal contexts, emphasizing that fact finders need to be mindful of the risks associated with both overaccepting false denials and accepting false reports.

The semantic structure of accuracy in eyewitness testimony

In two studies, we examined if correct and incorrect statements in eyewitness testimony differed in semantic content. Testimony statements were obtained from participants who watched staged crime films and were interviewed as eyewitnesses. We analyzed the latent semantic representations of these statements using LSA and BERT. Study 1 showed that the semantic space of correct statements differed from incorrect statements; correct statements were more closely related to a dominance semantic representation, whereas incorrect statements were more closely related to a communion semantic representation. Study 2 only partially replicated these findings, but a mega-analysis of the two datasets showed different semantic representations for correct and incorrect statements, with incorrect statements more closely related to representations of communion and abstractness. Given the critical role of eyewitness testimony in the legal context, and the generally low ability of fact-finders to estimate the accuracy of witness statements, our results strongly call for further research on semantic content in correct and incorrect testimony statements.

Credibility assessments of alibi accounts: the role of cultural intergroup bias

Research has shown that judges and jurors are influenced by suspect ethnicity and that they might discriminate against out-group suspects in making decisions. This study examined the tendency to favor in-group members, as predicted by social identity theory, in assessing alibi credibility. Forty Israeli-Jewish and 40 Israeli-Arab participants assessed the credibility of an alibi statement provided by a suspect who was either Israeli-Jewish or Israeli-Arab. Findings show that participants were more likely to believe the alibi when it was provided by an in-group suspect than by an out-group suspect, supporting intergroup bias in alibi credibility assessments. Practical and theoretical implications are discussed.

Norwegian judges' knowledge of factors affecting eyewitness testimony: a 12-year follow-up

Eyewitness evidence often plays a critical role in decisions made in the criminal justice system. To evaluate eyewitness testimony accurately, judges must be aware of factors that can contaminate this type of evidence. In 2008, a survey of judges in Norway revealed a lack of awareness of several factors that affect eyewitness testimony. In the current study, a survey was administered to Norwegian judges (N=98) to evaluate their knowledge of factors that affect eyewitness testimony. Results showed that judges' overall knowledge scores were similar to those reported in 2008, but substantial increases and decreases in knowledge were observed for specific factors. Additional analyses indicated that increased uncertainty regarding some eyewitness factors led to a decline in accuracy when compared to responses observed in 2008. The current study provides an updated assessment of judges' knowledge of eyewitness factors and highlights the need for more comprehensive training for judges regarding these factors.

Predicting Accuracy in Eyewitness Testimonies With Memory Retrieval Effort and Confidence

Evaluating eyewitness testimonies has proven a difficult task. Recent research, however, suggests that incorrect memories are more effortful to retrieve than correct memories, and confidence in a memory is based on retrieval effort. We aimed to replicate and extend these findings, adding retrieval latency as a predictor of memory accuracy. Participants watched a film sequence with a staged crime and were interviewed about its content. We then analyzed retrieval effort cues in witness responses. Results showed that incorrect memories included more "effort cues" than correct memories. While correct responses were produced faster than incorrect responses, delays in responses proved a better predictor of accuracy than response latency. Furthermore, participants were more confident in correct than incorrect responses, and the effort cues partially mediated this confidence-accuracy relation. In sum, the results support previous findings of a relationship between memory accuracy and objectively verifiable cues to retrieval effort.

True and false alibis among prisoners and their detection by police detectives

The present study was designed to determine whether differences exist between true and false alibis and how accurate police detectives and lay people are in determining the veracity of alibis. This article provides a replication of the research by Culhane et al. (2013) with more representative participants. In the first experiment, real suspects in a remand prison generated true or false alibis. In the second experiment, a subset of those alibis were written out and were provided to experienced police officers and students for alibi evaluation and discrimination. Our results show that differentiating between true and false alibis is difficult, and even when more representative materials and participants are included, the accuracy did not exceed 60%. Interestingly we found that students and police officers focus on other aspects during the alibi discrimination. Thus, research using student participant cannot be, directly, used in alibi discrimination studies.

Detecting children's lies: comparing true accounts about highly stressful injuries with unprepared, prepared, and coached lies

In this investigation, 514 university students judged whether children were telling the truth about highly emotional events. Eight children (half female, half 8-9 and the remainder 12-14 years old) had been injured seriously enough to require emergency room treatment and were interviewed a few days later. Each was yoked to three other children matched in age and gender who fabricated accounts under one of three conditions: lies that were unprepared, prepared (24 hours to prepare), and coached by parents. Participants were at chance when judging true accounts as well as unprepared and prepared lies. However, 74% of the coached lies were judged as true. Participants' confidence in their judgments, age, experience with children, and relevant coursework/training did not improve judgments.