Partitioning the sources of recognition confidence: The role of individual differences

Improving the diagnostic value of lineup rejections

Erroneous eyewitness identification evidence is likely the leading cause of wrongful convictions. To minimize this error, scientists recommend collecting confidence. Research shows that eyewitness confidence and accuracy are strongly related when an eyewitness identifies someone from an initial and properly administered lineup. However, confidence is far less informative of accuracy when an eyewitness identifies no one and rejects the lineup instead. In this study, I aimed to improve the confidence-accuracy relationship for lineup rejections in two ways. First, I aimed to find the lineup that yields the strongest confidence-accuracy relationship for lineup rejections by comparing the standard, simultaneous procedure used by police worldwide to the novel "reveal" procedure designed by scientists to boost accuracy. Second, I aimed to find the best method for collecting confidence. To achieve this secondary aim, I made use of machine-learning techniques to compare confidence expressed in words to numeric confidence ratings. First, I find a significantly stronger confidence-accuracy relationship for lineup rejections in the reveal than in the standard procedure regardless of the method used to collect confidence. Second, I find that confidence expressed in words captures unique diagnostic information about the likely accuracy of a lineup rejection separate from the diagnostic information captured by numeric confidence ratings. These results inform models of recognition memory and may improve the criminal-legal system by increasing the diagnostic value of a lineup rejection.

Unskilled, underperforming, or unaware? Testing three accounts of individual differences in metacognitive monitoring

Many studies show that competence (e.g., skill, expertise, natural ability) influences individuals' capabilities of monitoring their item-level performance. However, debate persists about how best to explain these individual differences in metacognition. The competence-based account ascribes differences in monitoring to individuals' objective ability level, arguing that the same skills necessary to perform a task are required to effectively monitor performance. The performance-based account attributes differences in monitoring to changes in overall task performance - no individual differences in competence required. Finally, the metacognitive awareness account proposes that alignment between an individuals' self-assessed and objective ability leads to differences in monitoring. In this study, 603 participants completed a self-assessment of face recognition ability, a lineup identification task, and an objective assessment of face recognition ability. We manipulated the number of encoding repetitions and delay between encoding and test to produce varying levels of task performance across objective face recognition ability. Following each lineup decision, participants provided both a numeric confidence rating and a written expression of verbal confidence. We transformed verbal confidence into a quantitative value using machine learning techniques. When matched on overall identification accuracy, objectively stronger face recognizers used numeric and verbal confidence that a) better discriminates between correct and filler lineup identifications than weaker recognizers, and b) shows better calibration to accuracy. Participants with greater self-assessed ability used higher levels of confidence, irrespective of trial accuracy. These results support the competence-based account.

Is It the Judge, the Sender, or Just the Individual Message? Disentangling Person and Message Effects on Variation in Lie-Detection Judgments

Research suggests that people differ more in their ability to lie than in their ability to detect lies. However, because studies have not treated senders and messages as separate entities, it is unclear whether some senders are generally more transparent than others or whether individual messages differ in their transparency of veracity regardless of senders. Variance attributable to judges, senders, and messages was estimated simultaneously using multiple messages from each sender (totaling more than 45,000 judgments). The claim that the accuracy of a veracity judgment depends on the sender was not supported. Messages differed in their detectability (21% explained variance), but senders did not. Message veracity accounted for most message variation (16.8% of the total variance), but other idiosyncratic message characteristics also contributed significantly. Consistent with the notion that a (mis)match between sender demeanor and veracity determines accuracy, lie and truth detectability differed individually within senders. Judges primarily determined variance in lie-versus-truth classifications (12%) and in confidence (46%) but played no role regarding judgment accuracy (< 0.01%). This work has substantial implications for the design and direction of future research and underscores the importance of separating senders and messages when developing theories and testing derived hypotheses.

Recognition, remember-know, and confidence judgments: no evidence of cross-contamination here!

ABSTRACTWe report three experiments designed to reveal the mechanisms that underlie subjective experiences of recognition by examining effects of how those experiences are measured. Prior research has explored the potential influences of collecting metacognitive measures on memory performance. Building on this work, here we systematically evaluated whether cross-measure contamination occurs when remember-know (RK) and/or confidence (C) judgments are made after old/new recognition decisions. In Experiment 1, making either RK or C judgments did not significantly influence recognition relative to a standard no-judgment condition. In Experiment 2, making RK judgments in addition to C judgments did not significantly affect recognition or confidence. In Experiment 3, making C judgments in addition to RK judgments did not significantly affect recognition or patterns of RK responses. Cross-contamination was not apparent regardless of whether items were studied using a shallow or deep levels-of-processing task - a manipulation that yielded robust effects on recognition, RK judgments, and C. Our results indicate that under some conditions, participants can independently evaluate their recognition, subjective recognition experience, and confidence. Though contamination across measures of metamemory and memory is always possible, it may not be inevitable. This has implications for the mechanisms that underlie subjective experiences that accompany recognition judgments.

Reexamining the sources of variance in recognition confidence: A reply to Kantner and Dobbins (2019)

Recently, Kantner and Dobbins (Psychonomic Bulletin & Review, 26(4), 1317-1324, 2019) reported a reanalysis of a series of previously published data sets in which they examined the sources of variation in recognition memory confidence ratings. Although between-subject differences in mean levels of confidence tended to account for the majority of variance in confidence ratings for both "old" and "new" decisions, the contribution of overall subject-level variation to confidence ratings for "new" decisions was noticeably and consistently larger. Here, I report a series of quantitative simulations along with a reanalysis of the original data to demonstrate that the relatively greater subject-level variation in mean confidence seen for "new" as compared with "old" recognition decisions largely reflects statistical constraints imposed by (a) the range limits of the ordinal scale used to measure confidence, and (b) the stronger relation between memory accuracy and confidence in "old" decisions. Therefore, any observed difference in the extent of subject-level variation in mean confidence between "old" and "new" recognition decisions need not imply a meaningful psychological distinction. These findings point out what in my view is an important statistical constraint that should be considered by researchers interested in understanding the bases of variation in memory confidence.

Accelerated Long-Term Forgetting: Prolonged Delayed Recognition as Sensitive Measurement for Different Profiles of Long-Term Memory and Metacognitive Confidence in Stroke Patients

OBJECTIVE

Deficits in episodic memory are frequently reported after ischemic stroke. In standard clinical care, episodic memory is assessed after a 20-30 min delay, with abnormal memory decay over this period being characterized as rapid forgetting (RF). Previous studies have shown abnormal forgetting over a prolonged interval (days to weeks) despite normal acquisition, referred to as accelerated long-term forgetting (ALF).

METHOD

We examined whether ALF is present in stroke patients (N = 91) using immediate testing (T1), testing after a short delay (20-30 min, T2), and testing after a prolonged delay (one week, T3). Based on performance compared to matched controls (N = 85), patients were divided into (1) patients without forgetting, (2) patients with RF between T1 and T2, and (3) patients with ALF at T3. Furthermore, confidence ratings were assessed.

RESULTS

ALF was present in a moderate amount of stroke patients (17%), but ALF was even more prevalent in our stroke sample than RF after a 20-30 min delay (which was found in only 13% of our patients). Patients reported a lower confidence for their responses, independent of their actual performance.

CONCLUSIONS

Adding a one-week delayed measurement may potentially assist in identifying patients with memory decrements that may otherwise go undetected.

Recognition language classifiers demonstrate far transfer of learning

Machine learners trained on verbal justifications of recognition decisions reliably predict recognition accuracy. If these recognition language classifiers are recollection sensitive, they should generalize beyond the single-item, verbal recognition paradigms upon which they were trained. To test this, three classifiers were trained to distinguish justification language in three different single-item verbal recognition paradigms, learning to distinguish the language justifying hits from false alarms, high from medium confidence hits, and remember from know judgements. The resulting classifiers were then used to predictively score language justifying correct versus incorrect eyewitness lineup selections constituting a test of far transfer because of the differences in materials (faces vs. words), subject populations (undergraduate vs. online), testing procedures (single vs. multiple items), and test lengths (12 vs. hundreds of targets per subject) among others. All three classifiers reliably predicted eyewitness accuracy despite these differences. Additionally, mixed modeling demonstrated that the classifiers demonstrated both convergent and divergent validity with respect to the recollection sensitivity hypothesis. That is, they strongly predicted the accuracy of eyewitness selections (i.e., hits vs. false alarms) but failed to predict the accuracy of eyewitness rejections (i.e., correct rejections vs. misses). Moreover, one classifier was shown to predict eyewitness confidence despite being trained on a design devoid of all metacognitive judgments. These findings support the hypothesis that recognition language classifiers detect recollection conveyed in the language subjects use to justify their memory decisions.

Applying confidence accuracy characteristic plots to old/new recognition memory experiments

Confidence-accuracy characteristic (CAC) plots were developed for use in eyewitness identification experiments, and previous findings show that high confidence indicates high accuracy in all studies of adults with an unbiased lineup. We apply CAC plots to standard old/new recognition memory data by calculating response-based and item-based accuracy, one using false alarms and the other using misses. We use both methods to examine the confidence-accuracy relationship for both correct old responses (hits) and new responses (correct rejections). We reanalysed three sets of published data using these methods and show that the method chosen, as well as the relation of lures to targets, determines the confidence-accuracy relation. Using response-based accuracy for hits, high confidence yields quite high accuracy, and this is generally true with the other methods, especially when lures are unrelated to targets. However, when analyzing correct rejections, the relationship between confidence and accuracy is less pronounced. When lures are semantically related to targets, the various CAC plots show different confidence-accuracy relations. The different methods of calculating CAC plots provide a useful tool in analyzing standard old/new recognition experiments. The results generally accord with unequal-variance signal detection models of recognition memory.

The Language of Recollection in Support of Recognition Memory Decisions

Abstract. Researchers often augment recognition memory decisions with confidence ratings or reports of “Remember” and “Know” experiences. While important, these ratings are subject to variation in interpretation and misspecification. Here we review recent findings from a “verbal reports as data” procedure in which subjects justify, in their own words, the basis of recognition. The application of a language pattern classifier to these justifications demonstrates that it: (a) is sensitive to the presence of recollection, (b) tracks individual differences in recognition accuracy, and (c) generalizes in a theoretically meaningful way to justifications from a separate experiment. More broadly, this approach should be useful for any cognitive decision task in which competing theories suggest different explicit bases underlying the judgments, or for which the explicit versus implicit basis of the decisions is in question.

Recognition memory: Tulving's contributions and some new findings

Endel Tulving has provided unparalleled contributions to the study of human memory. We consider here his contributions to the study of recognition memory and celebrate his first article on recognition, a nearly forgotten but (we argue) essential paper from 1968. We next consider his distinction between remembering and knowing, its relation to confidence, and the implications of high levels of false remembering in the DRM paradigm for using phenomenal experiences as measures of memory. We next pivot to newer work, the use of confidence accuracy characteristic plots in analyzing standard recognition memory experiments. We argue they are quite useful in such research, as they are in eyewitness research. For example, we report that even with hundreds of items, high confidence in a response indicates high accuracy, just as it does in one-item eyewitness research. Finally, we argue that amnesia (rapid forgetting) occurs in all people (not just amnesic patients) for some of their experiences. We provide evidence from three experiments revealing that subjects who fail to recognize recently studied items (miss responses) do so with high confidence 15-20% of the time. Such high confidence misses constitute our definition of everyday amnesia that can occur even in college student populations.