A Hierarchy of Expert Performance

Trust but verify: The biasing effects of witness opinions and background knowledge in workplace investigations

INTRODUCTION

A unique feature of workplace investigations is the familiarity that investigators and witnesses have with the factors involved in the adverse incident. Familiarity creates expectations that can shape investigators' and witnesses' assumptions and opinions. The current research examined the biasing effect of non-factual witness claims on investigators' judgments. These claims, which we call 'uncheckable,' included opinions about factors involved in the event and the future. We also examined how participants' a priori knowledge of an employee's history influenced their judgments.

METHOD

This experiment used a 2 (background information: control or unsafe) × 2 (uncheckable content: neutral or unsafe) between-subjects design. Participants were provided with background information about a worker (control or unsafe history) and a witness statement about a workplace event that contained uncheckable claims (neutral or worker as unsafe). We tested how our manipulations biased participants' judgments of: (i) the cause of the event, (ii) the witness's confidence and credibility, and (iii) the diagnosticity of the witness's account. We also tested if biasing background information affected how factual participants found the witness's statement.

RESULTS

Biasing uncheckable information (i.e., opinions) affected participants' judgments of event cause (ηp2 = 0.033) and increased their ratings of witness confidence (ηp2 = 0.074). Biasing background information about a worker affected participants' judgments of the cause of the event (ηp2 = 0.088), the diagnostic value of the witness statement (ηp2 = 0.054), and the number of factual claims in the witness statement, resulting in more uncheckable claims being misclassified as potential facts (ηp2 = 0.18).

CONCLUSION

This experiment demonstrated the significant effect that non-factual witness statements and irrelevant background information can have on the interpretation of evidence and judgments about the cause of events.

PRACTICAL APPLICATION

Understanding how contextual information can bias investigative judgment helps workplace investigators manage its influence in their judgment practices.

Experts' decision-making processes in Swedish forensic psychiatric investigations: A case vignette study

It has previously been demonstrated that decisions made by forensic experts can suffer from issues with both bias and poor reliability. The outcome of Swedish forensic psychiatric investigations can have a major impact on the courts' choice of sanction for a mentally disordered offender. These investigations are performed by multi-professional teams of experts, where each expert is obliged to state their opinion on whether the client has a severe mental disorder (SMD) or not. In the present study, a case vignette design was used to simulate the decision-making process of forensic psychiatric investigations. Of the 73 Swedish experts working with forensic psychiatric investigations, a total of 27 (37%) participated in the study. The results showed that the Swedish experts formulated multiple diagnostic hypotheses about cases throughout the process and revised these hypotheses when presented with new information. There was substantial variation between the experts in which hypotheses were seen as most relevant. While the experts grew more certain of their opinions on SMD during the simulated investigation, there was considerable variation in their opinions both throughout and at the end of the process. Although low statistical power and the sample not being randomized limit generalizations, the results indicate no idiosyncratic patterns in the decision-making processes of Swedish experts or signs of confirmation bias. If used properly, the variation in both process and outcome could be used to safeguard and possibly increase the reliability and validity of the final decision of Swedish forensic psychiatric investigations.

The most consistent finding in forensic science is inconsistency

The most consistent finding in many forensic science domains is inconsistency (i.e., lack of reliability, reproducibility, repeatability, and replicability). The lack of consistency is a major problem, both from a scientific and a criminal justice point of view. Examining forensic conclusion data, from across many forensic domains, highlights the underlying cognitive issues and offers a better understanding of the issues and challenges. Such insights enable the development of ways to minimize these inconsistencies and move forward. The aim is to highlight the problem, so that it can be minimized and the reliability of forensic science evidence can be improved.

The role of alternative hypotheses in reducing bias in forensic medical experts' decision making

Medical opinions are often essential evidence in criminal cases but relatively little is known about the factors that impact forensic doctors' decision making. This research examines the role and impact of having an alternative hypothesis while forming a medical opinion. A scenario-based experiment with forensic doctors (n = 20) was conducted. In two out of three scenarios, the existence of alternative hypotheses impacted the actual opinions reached, the confidence in the judgments and the perceived consistency with the plaintiff hypothesis. Investigative and legal actors should be aware of the possibility of biases and importance of having alternative hypotheses when requesting and evaluating medical opinions.

Work-related stress amongst legal medical doctors: the need for systematic psychological support. An Italian perspective

Legal medical doctors (LMDs) expertise encompasses a wide range of responsibilities. Work-related stress amongst LMDs is an extremely relevant factor, which affects the quality of LMDs life and work. Whilst it is a better-known problem abroad, this issue is poorly debated in Italy, as demonstrated by this literature analysis. The aim of this paper is to better understand the main sources of stress in the practice of legal medicine in Italy, highlighting the need for systematic psychological support. The risks of work-related stress for the social and health professions are well known in literature. In Italy, however, due to a series of complex circumstances and cultural and research delays, forensic practitioners often seem to be excluded from these kinds of complex issues. The authors, after a series of considerations drawn from a review of the literature and from experience in the forensic and medical field, point out how work in this area entails risks for workers, on par with all others belonging to the helping and social-health professions. They conclude their contribution with a series of proposals for appropriate protocols to cope with such problems for workers in the forensic and medical field.

Key points

Work-related risks for the social and health professionals are widely recognized in the literature.Although legal medical doctors are often thought to be safe from these types of issues, significant risks exist for this category as well.It is important to create appropriate protocols to address outcomes for legal medical doctors.

"Blood, Bucks and Bias": Reliability and biasability of crime scene investigators' selection and prioritization of blood traces

When crime scene investigators (CSIs) encounter crime scenes with large volumes of blood, some selection and prioritization is often needed, and this will impact on what blood is and is not available for forensic analysis. What factors influence CSIs decision making process is largely unknown. This study examines the effects of awareness of limited resources and irrelevant contextual case information indicating either a homicide or a suicide on CSIs collection of blood traces. To this end, two scenario-based experiments with CSIs and novices were conducted. Overall, the results suggest that even when CSIs decisions are made under identical conditions, their trace selection varies both when it comes to numbers and locations. Furthermore, awareness of limited resources made CSIs collect fewer traces and their selections also varied following the contextual case information, showing similarities and differences with novices. Since blood traces can be used to establish both activity and identity the findings can have important implications for the subsequent investigation as well as trial.

Cognitive bias in workplace investigation: Problems, perspectives and proposed solutions

Psychological research demonstrates how our perceptions and cognitions are affected by context, motivation, expectation, and experience. A mounting body of research has revealed the many sources of bias that affect the judgments of experts as they execute their work. Professionals in such fields as forensic science, intelligence analysis, criminal investigation, medical and judicial decision-making find themselves at an inflection point where past professional practices are being questioned and new approaches developed. Workplace investigation is a professional domain that is in many ways analogous to the aforementioned decision-making environments. Yet, workplace investigation is also unique, as the sources, magnitude, and direction of bias are specific to workplace environments. The workplace investigation literature does not comprehensively address the many ways that the workings of honest investigators' minds may be biased when collecting evidence and/or rendering judgments; nor does the literature offer a set of strategies to address such happenings. The current paper is the first to offer a comprehensive overview of the important issue of cognitive bias in workplace investigation. In it I discuss the abilities and limitations of human cognition, provide a framework of sources of bias, as well as, offer suggestions for bias mitigation in the investigation process.

Cognitive and human factors in legal layperson decision making: Sources of bias in juror decision making

Juries in adversarial courts are tasked with several responsibilities. They are asked to: 1) assess the credibility and reliability of the evidence presented; 2) deliberate; 3) and then reach a decision. Jurors are expected to evaluate said evidence in a rational/impartial manner, thus allowing the defendant their right to a fair trial. However, psychological research has shown that jurors are not rational and can reach inaccurate decisions by being biased by certain factors. The aim of the current review was to explore the potential sources from which biases are introduced into the jury. Three main sources of bias were focussed upon: 1) pre-trial bias; 2) cognitive bias; 3) bias from external legal actors (expert witnesses). Legal scholars commonly cite deliberations as a method of attenuating individual juror bias, this claim is evaluated in the review. The review concludes that bias is a multifaceted phenomenon introduced from many different elements, and that several sources of bias may interact with one another during a jury trial to cause the effects of bias to snowball. Four recommendations are made: 1) juror selection should be utilised to create heterogenous juries that challenge problematic biases from individual jurors; 2) increase the quality of expert testimony through training; 3) procedures such as Linear Sequential Unmasking should be adopted by expert witnesses to filter out some sources of bias; 4) legal professionals and jurors should be educated about the effects that biases may have on decision making; 5) more research into bias in jurors is needed.

Anthropometric Kit Development for Stunted Early Detection among Children Under-two Years Old: Providing a Portable Body Length Measurer

BACKGROUND: One of the keys to stunting reduction, a condition of lower height or length compared to their age, is the measurement of children in the community. However, the infantometer as the gold standard is not accessible by all community health workers (CHWs). AIM: The aim is to develop a stunted early detection tool (SEDT) for Indonesian children under-two years old. MATERIALS AND METHODS: This qualitative study was conducted as the first phase of the development process and focused on the experts’ judgments of the prototype. Experts’ judgments were recorded qualitatively. There were five in-depth interviews with anthropometric, health promotion, and media design experts. Rogers’ Diffusion of Innovations Theory and thematic content analysis were used to analyze the relative advantages, compatibility, complexity, and observability. RESULTS: The prototype of the SEDT consists of two tools, including a length mat to measure children’s length and a circular disc that helps CHWs classify the nutritional status of the children according to length for age length-for-age Z-score. Most experts agreed that the SEDT is a good instrument for the early detection of stunting among children under 24 months. The tool is designed to be portable, child-friendly, compatible, and easy to use. Although its development has the potential to help CHWs fulfill their responsibilities, major changes were needed specifically to improve the tool’s stability and design. CONCLUSIONS: This analysis gives broad information about the SEDT’s potential as a SEDT considering its relative advantages, complexity, compatibility, and observability. Further research is important to validate potential users’ responses in a representative population.

Decision-Making Within Forensic Psychiatric Investigations: The Use of Various Information Sources by Different Expert Groups to Reach Conclusions on Legal Insanity

BACKGROUND

Which type of information experts use to make decisions regarding legal insanity within forensic psychiatric investigations (FPI) is relatively unknown, both in general and when considering variations due to case context. It is important to explore this area to be able to counteract the effects of various kinds of cognitive bias.

METHOD

The aim was to explore whether FPI expert groups differed regarding case-specific as well as general use of information types required to make decisions on severe mental disorder (SMD). Three FPI case vignettes were presented to three professional groups involved in FPIs in Sweden (n = 41): forensic psychiatrists (n = 15), psychologists (n = 15), and social workers (n = 11). The participants reported which types of information they required to reach conclusions regarding SMD in each case. They also reported which types of information they had used within general FPI praxis during the previous year and the information types' perceived usefulness.

RESULTS

The expert groups differed somewhat regarding what type of information they required for the cases (e.g., results from cognitive testing), but some information was required in all cases (e.g., client's self-report). Regarding the preliminary assessment of SMD in the three cases, minor differences were found. Within the general FPI praxis, experts reported using several information types, while the general perceived usefulness of these sources varied.

DISCUSSION

The professional groups relied partly on a "core" of information sources, but some case-specific adaptations were found. The professional groups' inclination to suspect SMD also varied somewhat. This indicates a need to explore the potential consequences of these similarities and differences.

Metric forensic anthropology decisions: Reliability and biasability of sectioning-point-based sex estimates

Subjective decisions make human cognitive processes more susceptible to bias and error. Specifically, research indicates that additional context biases forensic anthropologists' morphological analyses. To address whether metric analyses are also subject to bias, we conducted a pilot study in which 52 experienced osteologists measured a difficult-to-classify human femur, with or without additional contextual information. Using a metric sectioning-point sex-estimation method, participants provided a sex estimate for individual skeletal element(s) and, when given multiple elements, the combined skeletal assemblage. Control group participants (n = 24) measured only the femur. In addition to the femur, bias group participants (n = 28) either measured a female humerus and viewed a female-biasing photograph (n = 14) or measured a male humerus and viewed a male-biasing photograph (n = 14). We explored whether the experts in the different groups would differ in: (1) femoral measurements; (2) femoral sex-estimation conclusions; and (3) final sex-estimation conclusions for the skeletal assemblage. Although the femoral measurements and femoral sex estimates were comparable across groups, the overall sex estimates in the female-biased group were impacted by contextual information-differing from both the control and male-biased groups (p < 0.001). Our results demonstrate that cognitive bias can occur even in metric sex-estimation conclusions. Specifically, this occurred when the metric data and single-element sex estimates were synthesized into an overall estimate. Thus, our results suggest that metric methods are most vulnerable to bias when data are synthesized into an overall conclusion, highlighting the need for bias countermeasures and comprehensive statistical frameworks for synthesizing metric data to mitigate the effects of cognitive bias.

Objectivity is a myth that harms the practice and diversity of forensic science

•

Forensic science data are theory laden; pure scientific objectivity is a myth.

•

Upholding this myth marginalizes forensic scientists with subjective positionalities

•

Objectivity rhetoric is exclusive; ethical forensic science needs diverse perspectives.

•

Espousing objectivity prevents us from supporting the communities we serve.

•

Mitigated objectivity acknowledges implicit bias, constraining it via quality control.

The biasing impact of irrelevant contextual information on forensic odontology radiograph matching decisions

The potential biasing effect of irrelevant context information on the forensic odontology method of radiograph-based identification has never been empirically investigated despite being a recognized problem in other forensic science disciplines. This study examines the effect of irrelevant context information on the probability judgment of match (JOM) of practicing forensic odontologist and dentist participants who were asked to match pairs of dental radiographs supplemented with irrelevant case information. The irrelevant case information contained domain task-irrelevant context information which varied in strength (strong or weak). It suggested either supportive or contradictory bias relative to the actual match status of the radiograph pairs. The dental radiographs consisted of verified match and non-match radiographs pairs sampled and de-identified from actual forensic cases. Changes in accuracy and JOM between supportive and contradictory contexts conditions revealed a contextual bias. Mixed model analysis showed that strong supportive context increased the odds ratio of correct decisions by a factor of 2.4 [1.23, 4.46]; p = 0.0097. Consistent with the biasing effect, the JOM score differences between strong supportive and contradictory irrelevant context information were 1.03 and 0.43 respectively for the non-match and match decisions. The direction of context suggestion (p = 0.0067), the radiograph match status (p = 0.014), and their interactions (p = 0.0061), were all found to impact the participants' decision. The weak context information was not strong enough to have a significant effect on accuracy or JOM scores. This study demonstrates that radiograph match judgment is affected and can be biased by strong irrelevant contextual information.

A hierarchy of expert performance as applied to forensic anthropology

Due to their medicolegal repercussions, forensic anthropology conclusions must be reliable, consistent, and minimally compromised by bias. Yet, a synthetic analysis of the reliability and biasability of the discipline's methods has not yet been conducted. To do so, this study utilized Dror's (2016) hierarchy of expert performance (HEP), an eight-level model aimed at examining intra- and inter-expert reliability and biasability (the potential for cognitive bias) within the literature of forensic science disciplines. A systematic review of the forensic anthropology literature was conducted (1972-present), including papers published in the American Journal of Physical Anthropology, Forensic Anthropology, Forensic Science International, and the Journal of Forensic Sciences and Anthropology Section abstracts published in the Proceedings of the Annual Meeting of the AAFS which matched keywords such as "forensic anthropology," "bias," "reliability," "cognition," "cognitive," or "error." The resulting forensic anthropology HEP showcases areas that have ample research and areas where more research can be conducted. Specifically, statistically significant increases in reliability (p < 0.001) and biasability (p < 0.001) publications were found since 2009 (publication of the NAS report). Extensive research examined the reliability of forensic anthropological observations and conclusions (n = 744 publications). However, minimal research investigated the biasability of forensic anthropological observations and conclusions (n = 20 publications). Notably, while several studies demonstrated the biasing effect of extraneous information on anthropological morphological assessments, there was no research into these effects on anthropological metric assessments. The findings revealed by the forensic anthropology HEP can help to guide future research, ultimately informing the development and refinement of best-practice standards for the discipline.

The effect of contextual information on professional judgment: Reliability and biasability of expert workplace safety inspectors

INTRODUCTION

A critical aspect of occupational safety is workplace inspections by experts, in which hazards are identified. Scientific research demonstrates that expectation generated by context (i.e., prior knowledge and experience) can bias the judgments of professionals and that individuals are largely unaware when their judgments are affected by bias.

METHOD

The current research tested the reliability and biasability of expert safety inspectors' judgments. We used a two-study design (Study 1, N = 83; Study 2, N = 70) to explore the potential of contextual, task-irrelevant, information to bias professionals' judgments. We examined three main issues: (1) the effect that biasing background information (safe and unsafe company history) had on professional regulatory safety inspectors' judgments of a worksite; (2) the reliability of those judgments amongst safety inspectors and (3) inspectors' awareness of bias in their judgments and confidence in their performance.

RESULTS

Our findings establish that: (i) inspectors' judgments were biased by historical contextual information, (ii) they were not only biased, but the impact was implicit: they reported being unaware that it affected their judgments, and (iii) independent of our manipulations, inspectors were inconsistent with one another and the variations were not a product of experience.

CONCLUSION

Our results are a replication of findings from a host of other professional domains, where honest, hardworking professionals underappreciate the biasing effect of context on their decision making. The current paper situates these findings within the relevant research on safety inspection, cognitive bias and decision making, as well as provides suggestions for bias mitigation in workplace safety inspection. Practical Application: Our results have implications for occupational health and safety given that inspection is an integral aspect of an effective safety system. In addition to our findings, this study contributes to the literature by providing recommendations regarding how to mitigate the effect of bias in inspection.

Are forensic scientists too risk averse?

Fingerprint examiners maintain decision thresholds that represent the amount of evidence required for an identification or exclusion conclusion. As measured by error rate studies (Proc Natl Acad Sci USA. 2011;108(19):7733-8), these decision thresholds currently exhibit a preference for preventing erroneous identification errors at the expense of preventing erroneous exclusion errors. The goal of this study is to measure the decision thresholds for both fingerprint examiners and members of the general public, to determine whether examiners are more risk averse than potential jury members. To externally measure these decision thresholds, subjects manipulated decision criteria in a web-based visualization that reflects the trade-offs between erroneous identification decisions and erroneous exclusion decisions. Data from fingerprint examiners and the general public were compared to determine whether both groups have similar values as expressed by the placement of the decision criteria. The results of this study show that fingerprint examiners are more risk averse than members of the general public, although they align with error rate studies of fingerprint examiners. Demographic data demonstrate those factors that may contribute to differences in decision criterion placement, both between the two groups and between individuals within a group. The experimental methods provide a rich framework for measuring, interpreting, and responding to the values of society as applied to forensic decision-making.

Cognitive bias in forensic pathology decisions

Forensic pathologists’ decisions are critical in police investigations and court proceedings as they determine whether an unnatural death of a young child was an accident or homicide. Does cognitive bias affect forensic pathologists’ decision‐making? To address this question, we examined all death certificates issued during a 10‐year period in the State of Nevada in the United States for children under the age of six. We also conducted an experiment with 133 forensic pathologists in which we tested whether knowledge of irrelevant non‐medical information that should have no bearing on forensic pathologists’ decisions influenced their manner of death determinations. The dataset of death certificates indicated that forensic pathologists were more likely to rule "homicide" rather than "accident" for deaths of Black children relative to White children. This may arise because the base‐rate expectation creates an a priori cognitive bias to rule that Black children died as a result of homicide, which then perpetuates itself. Corroborating this explanation, the experimental data with the 133 forensic pathologists exhibited biased decisions when given identical medical information but different irrelevant non‐medical information about the race of the child and who was the caregiver who brought them to the hospital. These findings together demonstrate how extraneous information can result in cognitive bias in forensic pathology decision‐making.

Translational Application of a Neuro-Scientific Multi-Modal Approach Into Forensic Psychiatric Evaluation: Why and How?

A prominent body of literature indicates that insanity evaluations, which are intended to provide influential expert reports for judges to reach a decision "beyond any reasonable doubt," suffer from a low inter-rater reliability. This paper reviews the limitations of the classical approach to insanity evaluation and the criticisms to the introduction of neuro-scientific approach in court. Here, we explain why in our opinion these criticisms, that seriously hamper the translational implementation of neuroscience into the forensic setting, do not survive scientific scrutiny. Moreover, we discuss how the neuro-scientific multimodal approach may improve the inter-rater reliability in insanity evaluation. Critically, neuroscience does not aim to introduce a brain-based concept of insanity. Indeed, criteria for responsibility and insanity are and should remain clinical. Rather, following the falsificationist approach and the convergence of evidence principle, the neuro-scientific multimodal approach is being proposed as a way to improve reliability of insanity evaluation and to mitigate the influence of cognitive biases on the formulation of insanity opinions, with the final aim to reduce errors and controversies.

Expert Reliability in Legal Proceedings: "Eeny, Meeny, Miny, Moe, With Which Expert Should We Go?"

Between Expert Reliability refers to the extent to which different experts examining identical evidence make the same observations and reach the same conclusions. Some areas of expert decision making have been shown to entail questions with relatively low Between Expert Reliability, but the disagreement between experts is not always communicated to the legal actors forming decisions on the basis of the expert evidence. In this paper, we discuss the issues of Between Expert Reliability in legal proceedings, using forensic age estimations as a case study. Across national as well international jurisdictions, there is large variation in which experts are hired to conduct age estimations as well as the methods they use. Simultaneously, age estimations can be fully decisive for outcomes e.g. in asylum law and criminal law. Using datasets obtained from the Swedish legal context, we identify that radiologists and odontologists examining knees or teeth images to estimate age seem to disagree within their own disciplines (radiologist 1 v. radiologist 2 or odontologist 1 v. odontologist 2) as well as across different disciplines (radiologist v. odontologist) relatively often. This may have large implications e.g. in cases where only one expert from the respective field is involved. The paper discusses appropriate ways for legal actors to deal with the possibility of lacking Between Expert Reliability. This is indeed a challenging task provided that legal actors are legal experts but not necessarily scientific experts.

Biasability and reliability of expert forensic document examiners

The performance of experts can be characterized in terms of biasability and reliability of their judgments. The current research is the first to explore the judgments of practicing forensic document experts, professionals who examine and compare disputed handwritten evidence to handwriting exemplars of individuals involved in criminal or civil litigation. Forensic handwriting experts determine if questioned and known handwritten items are of common authorship or written by different individuals, and present their findings in legal proceedings. The expert participants in our study (N=25) were not aware that they were part of a research study. Thirteen participants were led to believe that they were working on a case commissioned from the prosecution and the other twelve that it was for the defense. We did not find evidence in this study that this information biased their judgments, which may make sense since document examiners (in contrast to many other forensic domains) do not primarily work within an organizational forensic laboratory culture. The lack of bias in our findings may have been also due to the stimuli we used or/and the great variability in the judgments within each group, reflecting a lack of consistency in conclusions among examiners. A detailed discussion of our findings is presented along with the limitations that may have affected our results.

Limited not lazy: a quasi-experimental secondary analysis of evidence quality evaluations by those who hold implausible beliefs

Past research suggests that an uncritical or ‘lazy’ style of evaluating evidence may play a role in the development and maintenance of implausible beliefs. We examine this possibility by using a quasi-experimental design to compare how low- and high-quality evidence is evaluated by those who do and do not endorse implausible claims. Seven studies conducted during 2019–2020 provided the data for this analysis (N = 746). Each of the seven primary studies presented participants with high- and/or low-quality evidence and measured implausible claim endorsement and evaluations of evidence persuasiveness (via credibility, value, and/or weight). A linear mixed-effect model was used to predict persuasiveness from the interaction between implausible claim endorsement and evidence quality. Our results showed that endorsers were significantly more persuaded by the evidence than non-endorsers, but both groups were significantly more persuaded by high-quality than low-quality evidence. The interaction between endorsement and evidence quality was not significant. These results suggest that the formation and maintenance of implausible beliefs by endorsers may result from less critical evidence evaluations rather than a failure to analyse. This is consistent with a limited rather than a lazy approach and suggests that interventions to develop analytical skill may be useful for minimising the effects of implausible claims.

A multidisciplinary approach to insanity assessment as a way to reduce cognitive biases

Insanity assessment requires the evaluation of the psychopathological condition that underlies the mens rea. Psychopathological evaluation may be quite challenging due to (i) absence of biomarkers; (ii) low inter-rater reliability; (iii) presence of cognitive bias. This intrinsic low reliability of forensic psychiatric diagnosis does impact on insanity assessment, leading to arbitrary and unjust legal outcomes for the examinee. Thus, strategies to improve the reliability of insanity evaluation are strongly needed. A multidisciplinary approach has been proposed as a way to enrich clinical diagnosis with reliable and biologically founded data, thus minimizing subjectivity, reducing controversies and increasing inter-subject concordance in insanity assessment. By discussing a real case, here we show how the convergence of multiple indices can produce evidence that cannot be denied without introducing logical fallacies. Applying this approach, the forensic discussion will move from the presence/absence of psychopathology to the impact of psychopathology on insanity. This article illustrates how a multidisciplinary evaluation, which integrates neuroscientific methods with the classical insanity assessment, may lead to a more accurate approach in insanity evaluation. Critically, this approach will minimize the impact of cognitive bias on insanity opinion and thus result in an improvement of the whole criminal justice process.

INFERENCE: An Evidence-Based Approach for Medicolegal Causal Analyses

A fundamental purpose of forensic medical, or medicolegal, analysis is to provide legal factfinders with an opinion regarding the causal relationship between an alleged unlawful or negligent action and a medically observed adverse outcome, which is needed to establish legal liability. At present, there are no universally established standards for medicolegal causal analysis, although several different approaches to causation exist, with varying strengths and weaknesses and degrees of practical utility. These approaches can be categorized as intuitive or probabilistic, which are distributed along a spectrum of increasing case complexity. This paper proposes a systematic approach to evidence-based assessment of causation in forensic medicine, called the INtegration of Forensic Epidemiology and the Rigorous EvaluatioN of Causation Elements (INFERENCE) approach. The INFERENCE approach is an evolution of existing causal analysis methods and consists of a stepwise method of increasing complexity. We aimed to develop a probabilistic causal analysis approach that (1) fits the needs of legal factfinders who require an estimate of the probability of causation, and (2) is still sufficiently straightforward to be applied in real-world forensic medical practice. As the INFERENCE approach is most relevant in complex cases, we also propose a process for selecting the most appropriate causal analysis method for any given case. The goal of this approach is to improve the reproducibility and transparency of causal analyses, which will promote evidence-based practice and quality assurance in forensic medicine, resulting in expert opinions that are reliable and objective in legal proceedings.

Amplifying Domain Expertise in Clinical Data Pipelines

Digitization of health records has allowed the health care domain to adopt data-driven algorithms for decision support. There are multiple people involved in this process: a data engineer who processes and restructures the data, a data scientist who develops statistical models, and a domain expert who informs the design of the data pipeline and consumes its results for decision support. Although there are multiple data interaction tools for data scientists, few exist to allow domain experts to interact with data meaningfully. Designing systems for domain experts requires careful thought because they have different needs and characteristics from other end users. There should be an increased emphasis on the system to optimize the experts' interaction by directing them to high-impact data tasks and reducing the total task completion time. We refer to this optimization as amplifying domain expertise. Although there is active research in making machine learning models more explainable and usable, it focuses on the final outputs of the model. However, in the clinical domain, expert involvement is needed at every pipeline step: curation, cleaning, and analysis. To this end, we review literature from the database, human-computer information, and visualization communities to demonstrate the challenges and solutions at each of the data pipeline stages. Next, we present a taxonomy of expertise amplification, which can be applied when building systems for domain experts. This includes summarization, guidance, interaction, and acceleration. Finally, we demonstrate the use of our taxonomy with a case study.

Firearm examination: Examiner judgments and computer-based comparisons

Forensic firearm examination provides the court of law with information about the source of fired cartridge cases. We assessed the validity of source decisions of a computer-based method and of 73 firearm examiners who compared breechface and firing pin impressions of 48 comparison sets. We also compared the computer-based method's comparison scores with the examiners' degree-of-support judgments and assessed the validity of the latter. The true-positive rate (sensitivity) and true-negative rate (specificity) of the computer-based method (for the comparison of both the breechface and firing pin impressions) were 94.4% and at least 91.7%, respectively. For the examiners, the true-positive rate was at least 95.3% and the true-negative rate was at least 86.2%. The validity of the source decisions improved when the evaluations of breechface and firing pin impressions were combined and for the examiners also when the perceived difficulty of the comparison decreased. The examiners were reluctant to provide source decisions for "difficult" comparisons even though their source decisions were mostly correct. The correlation between the computer-based method's comparison scores and the examiners' degree-of-support judgments was low for the same-source comparisons to negligible for the different-source comparisons. Combining the outcomes of computer-based methods with the judgments of examiners could increase the validity of firearm examinations. The examiners' numerical degree-of-support judgments for their source decisions were not well-calibrated and showed clear signs of overconfidence. We suggest studying the merits of performance feedback to calibrate these judgments.

Microscopical discrimination of human head hairs sharing a mitochondrial haplogroup

In forensic analyses, determining the level of consensus among examiners for hair comparison conclusions and ancestry identifications is important for assessing the scientific validity of microscopical hair examinations. Here, we present data from an interlaboratory study on the accuracy of microscopical hair comparisons among a subset of experienced hair examiners currently analyzing hair in forensic laboratories across the United States. We examined how well microscopical analysis of hair can reliably be used to differentiate hair samples, many of which were macroscopically similar. Using cut hair samples, many sharing similar macroscopic and microscopic features, collected from individuals who share the same mitochondrial haplogroup as an indication of genetic relatedness, we tested multiple aspects that could impact hair comparisons. This research tested the extent to which morphological features related to ancestry and hair length influence conclusions. Microscopical hair examinations yielded accurate assessments of inclusion/exclusion relative to the reference samples among 85% of the pairwise comparisons. We found shorter hairs had reduced levels of accuracy and hairs from populations examiners were not familiar with may have impacted their ability to resolve features. The reliability of ancestry determinations is not yet clear, but we found indications that the existing categories are only somewhat related to current ethnic and genetic variation. Our results provide support for the continued utility of microscopical comparison of hairs within forensic laboratories and to advocate for a combined analytical approach using both microscopical analysis and mtDNA data on all forensic analyses of hair.

Organizational and Human Factors Affecting Forensic Decision-Making: Workplace Stress and Feedback

Although forensic examiners operate in a stressful environment, there is a lack of understanding about workplace stress and feedback. These organizational and human factors can potentially impact forensic science judgments. In this study, 150 practicing forensic examiners from one laboratory were surveyed about their experiences of workplace stress, and the explicit and implicit feedback they receive. Forensic examiners reported that their high stress levels originated more from workplace-related factors (management and/or supervision, backlogs, and the pressure to do many cases) than from personal related factors (family, medical, and/or financial). The findings showed that a few (8%) of the forensic examiners sometimes felt strong implicit feedback about what conclusions were expected from them and that some (14%) also strongly felt that they were more appreciated when they helped to solve a case (e.g., by reaching a "match" as opposed to an "inconclusive" conclusion). Differences were found when comparing workplace stress and feedback levels across three core forensic science fields (forensic biology, chemistry, and latent prints) and across career stages (early, mid, and late). Gaining insights into the stress factors within a workplace and explicit and implicit feedback has implications for developing policies to improve the well-being, motivation, and performance of forensic examiners.

Applying Hierarchy of Expert Performance (HEP) to investigative interview evaluation: strengths, challenges and future directions

The purpose of this paper is to systematically examine the research literature on the decision of expert interviewers within the theoretical framework of the Hierarchy of Expert Performance (HEP). After providing an overview of the HEP framework, existing research in the investigative interviewing at each of the eight levels of the HEP framework is reviewed. The results identify areas of strength in reliability between experts' observations (Level 2) and of weakness in reliability between experts' conclusions (Level 6). Biases in investigative interview experts' decision making is also revealed at biasability between expert conclusions (Level 8). Moreover, no published data are available in reliability within experts at the level of observations (Level 1) or conclusions (Level 5), biasability within or between expert observations (Level 3 and 4) and biasability within expert conclusions (Level 7). The findings highlight areas where future research and practical endeavour are much needed for the investigative interview.

Cognitive and Human Factors in Expert Decision Making: Six Fallacies and the Eight Sources of Bias

Fallacies about the nature of biases have shadowed a proper cognitive understanding of biases and their sources, which in turn lead to ways that minimize their impact. Six such fallacies are presented: it is an ethical issue, only applies to "bad apples", experts are impartial and immune, technology eliminates bias, blind spot, and the illusion of control. Then, eight sources of bias are discussed and conceptualized within three categories: (A) factors that relate to the specific case and analysis, which include the data, reference materials, and contextual information, (B) factors that relate to the specific person doing the analysis, which include past experience base rates, organizational factors, education and training, and personal factors, and lastly, (C) cognitive architecture and human nature that impacts all of us. These factors can impact what the data are (e.g., how data are sampled and collected, or what is considered as noise and therefore disregarded), the actual results (e.g., decisions on testing strategies, how analysis is conducted, and when to stop testing), and the conclusions (e.g., interpretation of the results). The paper concludes with specific measures that can minimize these biases.

The Error in "Error Rate": Why Error Rates Are So Needed, Yet So Elusive

Establishing error rates is crucial for knowing how well one is performing, determining whether improvement is needed, measuring whether interventions are effective, as well as for providing transparency. However, the flurry of activities in establishing error rates for the forensic sciences has largely overlooked some fundamental issues that make error rates a problematic construct and limit the ability to obtain a meaningful error rate. These include knowing the ground truth, establishing appropriate databases, determining what counts as an error, characterizing what is an acceptable error rate, ecological validity, and transparency within the adversarial legal system. Without addressing these practical and theoretical challenges, the very notion of a meaningful error rate is limited.

Assessing the reliability of a clothing-based forensic identification

Our justice system relies critically on the use of forensic science. More than a decade ago, a highly critical report raised significant concerns as to the reliability of many forensic techniques. These concerns persist today. Of particular concern to us is the use of photographic pattern analysis that attempts to identify an individual from purportedly distinct features. Such techniques have been used extensively in the courts over the past half century without, in our opinion, proper validation. We propose, therefore, that a large class of these forensic techniques should be subjected to rigorous analysis to determine their efficacy and appropriateness in the identification of individuals.

A 2009 report by the National Academy of Sciences was highly critical of many forensic practices. This report concluded that significant changes and advances were required to ensure the reliability across the forensic sciences. We examine the reliability of one such forensic technique used for identification based on purported distinct patterns on the seams of denim pants. Although first proposed more than 20 years ago, no thorough analysis of reliability or reproducibility of this forensic technique has previously been reported. We performed a detailed analysis of this forensic technique to determine its reliability and efficacy.

Cumulative Disadvantage: A Psychological Framework for Understanding How Innocence Can Lead to Confession, Wrongful Conviction, and Beyond

False confessions are a contributing factor in almost 30% of DNA exonerations in the United States. Similar problems have been documented all over the world. We present a novel framework to highlight the processes through which innocent people, once misidentified as suspects, experience cumulative disadvantages that culminate in pernicious consequences. The cumulative-disadvantage framework details how the innocent suspect's naivete and the interrogator's presumption of guilt trigger a process that can lead to false confession, the aftereffects of which spread to corrupt evidence gathering, bias forensic analysis, and virtually ensure wrongful convictions at trial or through pressured false guilty pleas. The framework integrates nascent research underscoring the enduring effects of the accumulated disadvantages postconviction and even after exoneration. We synthesize findings from psychological science, corroborating naturalistic evidence, and relevant legal precedents to explain how an innocent suspect's disadvantages can accumulate through the actions of law enforcement, forensic examiners, prosecutors, defense attorneys, judges, juries, and appeals courts. We conclude with prescribed research directions that can lead to empirically driven reforms to address the gestalt of the multistage process.

(Mis)use of scientific measurements in forensic science

Forensic science error rate studies have not given sufficient attention or weight to inconclusive evidence and inconclusive decisions. Inconclusive decisions can be correct decisions, but they can also be incorrect decisions. Errors can occur when inconclusive evidence is determined as an identification or exclusion, or conversely, when same- or different-source evidence is incorrectly determined as inconclusive. We present four common flaws in error rate studies: 1. Not including test items which are more prone to error; 2. Excluding inconclusive decisions from error rate calculations; 3. Counting inconclusive decisions as correct in error rate calculations; and 4. Examiners resorting to more inconclusive decisions during error rate studies than they do in casework. These flaws seriously undermine the credibility and accuracy of error rates reported in studies. To remedy these shortcomings, we present the problems and show the way forward by providing a corrected experimental design that quantifies error rates more accurately.

Perceptions and estimates of error rates in forensic science: A survey of forensic analysts

Every scientific technique features some error, and legal standards for the admissibility of scientific evidence (e.g., Daubert v. Merrill Dow Pharmaceuticals, Inc., 1993; Kumho Tire Co v. Carmichael, 1999) guide trial courts to consider known error rates. However, recent reviews of forensic science conclude that error rates for some common techniques are not well-documented or even established (e.g., NAS, 2009; PCAST, 2016). Furthermore, many forensic analysts have historically denied the presence of error in their field. Therefore, it is important to establish what forensic scientists actually know or believe about errors rates in their disciplines. We surveyed 183 practicing forensic analysts to examine what they think and estimate about error rates in their various disciplines. Results revealed that analysts perceive all types of errors to be rare, with false positive errors even more rare than false negatives. Likewise, analysts typically reported that they prefer to minimize the risk of false positives over false negatives. Most analysts could not specify where error rates for their discipline were documented or published. Their estimates of error in their fields were widely divergent - with some estimates unrealistically low.

The charm of structural neuroimaging in insanity evaluations: guidelines to avoid misinterpretation of the findings

Despite the popularity of structural neuroimaging techniques in twenty-first-century research, its results have had limited translational impact in real-world settings, where inferences need to be made at the individual level. Structural neuroimaging methods are now introduced frequently to aid in assessing defendants for insanity in criminal forensic evaluations, with the aim of providing "convergence" of evidence on the mens rea of the defendant. This approach may provide pivotal support for judges' decisions. Although neuroimaging aims to reduce uncertainty and controversies in legal settings and to increase the objectivity of criminal rulings, the application of structural neuroimaging in forensic settings is hampered by cognitive biases in the evaluation of evidence that lead to misinterpretation of the imaging results. It is thus increasingly important to have clear guidelines on the correct ways to apply and interpret neuroimaging evidence. In the current paper, we review the literature concerning structural neuroimaging in court settings with the aim of identifying rules for its correct application and interpretation. These rules, which aim to decrease the risk of biases, focus on the importance of (i) descriptive diagnoses, (ii) anatomo-clinical correlation, (iii) brain plasticity and (iv) avoiding logical fallacies, such as reverse inference. In addition, through the analysis of real forensic cases, we describe errors frequently observed due to incorrect interpretations of imaging. Clear guidelines for both the correct circumstances for introducing neuroimaging and its eventual interpretation are defined.

Faith in thy threshold

The current study focussed on the decision-making processes of jurors. The study investigated how jurors make a decision, if they integrate information within their decision-making process and if cue utilisation thresholds promote confirmation bias. To do this, 108 participants listened to one of nine cases. These participants were asked to give a likelihood of guilt rating after each piece of evidence, to state what the last piece of information was that they needed to make a decision and to give a final verdict at the end of a trial. The results highlighted that threshold decision making was being utilised, that information integration may allow thresholds to be reached and that thresholds may promote confirmation bias to reduce cognitive dissonance. In conclusion, this suggests that jurors integrate information until they reach a leading verdict, then the evaluation of information is distorted to support the leading threshold. Implications relate to legal instructions for jurors.

Work in progress: the progression of competence-based employability

Purpose

Employability and its components have received a lot of attention from scholars and practitioners. However, little is known about the interrelations between these different components of employability and how employees progress within their employability trajectories. Therefore, a model of such progression was constructed and tested using Van der Heijde and Van der Heijden’s (2006) employability measurement instrument. The paper aims to discuss these issues.

Design/methodology/approach

The propositions were tested empirically by applying a Rasch model using a sample of 167 Austrian business consultants.

Findings

The findings lend some support for the hypothesized progression model of employability. Specifically, the items measuring occupational expertise are largely located in the group of items that were relatively likely to be endorsed. Also, the items of personal flexibility and anticipation and optimization were, in general, less likely to be endorsed than the items of occupational expertise.

Research limitations/implications

The major thrust of this paper is a theoretical one. However, the empirical demonstration tentatively supports the proposed model, which implies that further, more robust longitudinal research in this direction may be a worthwhile endeavor.

Practical implications

By understanding which competences are important at which stage or across which stages of an individual’s career, career advisors and human resource management professionals can give more targeted advice concerning career management practices.

Originality/value

The present study contributes to the literature by investigating how employees may make progress within their employability trajectories.

Diagnosing Crime and Diagnosing Disease-II: Visual Pattern Perception and Diagnostic Accuracy

Previously, we reviewed how general cognitive processes might be susceptible to bias across both forensic and clinical fields, and how interdisciplinary comparisons could reduce error. We discuss several examples of clinical tasks which are heavily dependent on visual processing, comparing them to eyewitness identification (EI). We review the "constructive" nature of visual processing, and how contextual factors influence both medical experts and witnesses in decision making and recall. Overall, studies suggest common cognitive factors uniting these visual tasks, in both their strengths and shortcomings. Recently forensic sciences have advocated reducing errors by identifying and controlling nonrelevant information. Such efforts could effectively assist medical diagnosis. We suggest potential remedies for cognitive bias in these tasks. These can generalize across the clinical and forensic domains, including controlling the sequencing of contextual factors. One solution is an agnostic primary reading before incorporation of a complete history and interpretation.

Minds treating brains: understanding the interpretation of non-contrast CT ASPECTS in acute ischemic stroke

INTRODUCTION

The Alberta Stroke Program Early CT Score on non-contrast CT is a key component of prognostication and treatment selection in acute stroke care. Previous findings show that the reliability of this scale must be improved to maximize its clinical utility. Areas covered: This review discusses technical, patient-level, and reader-level sources of variability in ASPECTS reading; relevant concepts in the psychology of medical image perception; and potential interventions likely to improve inter- and intra-rater reliability. Expert commentary: Approaching variability in medical decision making from a psychological perspective will afford cognitively informed insights into the development of interventions and training techniques aimed at improving this issue.