A dual-process account of the development of scientific reasoning: The nature and development of metacognitive intercession skills

Generalization Bias in Science

Many scientists routinely generalize from study samples to larger populations. It is commonly assumed that this cognitive process of scientific induction is a voluntary inference in which researchers assess the generalizability of their data and then draw conclusions accordingly. We challenge this view and argue for a novel account. The account describes scientific induction as involving by default a generalization bias that operates automatically and frequently leads researchers to unintentionally generalize their findings without sufficient evidence. The result is unwarranted, overgeneralized conclusions. We support this account of scientific induction by integrating a range of disparate findings from across the cognitive sciences that have until now not been connected to research on the nature of scientific induction. The view that scientific induction involves by default a generalization bias calls for a revision of the current thinking about scientific induction and highlights an overlooked cause of the replication crisis in the sciences. Commonly proposed interventions to tackle scientific overgeneralizations that may feed into this crisis need to be supplemented with cognitive debiasing strategies against generalization bias to most effectively improve science.

Causal theory error in college students' understanding of science studies

When reasoning about science studies, people often make causal theory errors by inferring or accepting a causal claim based on correlational evidence. While humans naturally think in terms of causal relationships, reasoning about science findings requires understanding how evidence supports—or fails to support—a causal claim. This study investigated college students’ thinking about causal claims presented in brief media reports describing behavioral science findings. How do science students reason about causal claims from correlational evidence? And can their reasoning be improved through instruction clarifying the nature of causal theory error? We examined these questions through a series of written reasoning exercises given to advanced college students over three weeks within a psychology methods course. In a pretest session, students critiqued study quality and support for a causal claim from a brief media report  suggesting an association between two variables. Then, they created diagrams depicting possible alternative causal theories. At the beginning of the second session, an instructional intervention introduced students to an extended example of a causal theory error through guided questions about possible alternative causes. Then, they completed the same two tasks with new science reports immediately and again 1 week later. The results show students’ reasoning included fewer causal theory errors after the intervention, and this improvement was maintained a week later. Our findings suggest that interventions aimed at addressing reasoning about causal claims in correlational studies are needed even for advanced science students, and that training on considering alternative causal theories may be successful in reducing casual theory error.

Age, numeracy, and cultural differences in Chinese and American adolescents' performance on the ratio bias task

Ratio bias occurs when low-probability events with large numerators are judged as more probable than identical or higher-probability ratios with small numerators. Chinese and American adolescents made judgments on ratio bias problems with identical winning probabilities and unequal winning probabilities and completed a test of numeracy. In general, older participants performed better than younger participants and Chinese participants performed subtly better than American participants. On both problem types, numeracy mediated the relationships between age and performance. Between-country differences on unequal ratio problems were moderated by numeracy, such that Chinese participants outperformed American adolescents only at the most numerate level. By contrast, numeracy neither mediated nor moderated the country-performance relationship on equal ratio problems; instead, Chinese adolescents outperformed American adolescents at each level of numeracy, although these differences were fairly small. Discussion focuses on transferring formal math skills to everyday judgments, the associations among age, culture, and numeracy from a dual-process perspective, and limitations and alternative interpretations of the findings.

Developmental frontal brain activation differences in overcoming heuristic bias

Since reasoning is often biased by intuitive heuristics, the development of sound reasoning has long been postulated to depend on successful bias monitoring and inhibition. The present fMRI study aimed to identify neural correlates of developmental changes in these processes. A group of adults and young adolescents were presented with ratio-bias problems in which an intuitively cued heuristic response could be incongruent (conflict item) or congruent (no-conflict item) with the correct response. Results showed that successfully avoiding biased responding on conflict items across both age groups was associated with increased activation in Anterior Cingulate Cortex (ACC) and the right Lateral Prefrontal Cortex (LPFC) regions of interest. Critically, the right LPFC activation decreased with age. Biased responding did not result in right LPFC or ACC modulation and failed to show any developmental activation changes. We discuss implications for ongoing debates on the nature of heuristic bias and its development.

Effects of Thinking Dispositions, General Ability, Numeracy, and Instructional Set on Judgments and Decision-Making

To explore hypotheses based on Stanovich’s proposal that analytic processing comprises a reflective-level, an algorithmic level, and specific mindware, 342 participants completed measures of thinking dispositions, general ability (GA), numeracy, and probabilistic and nonprobabilistic reasoning. In a control condition, numeracy predicted probabilistic reasoning at high levels of both thinking dispositions and GA, and GA predicted nonprobabilistic reasoning at high levels of thinking dispositions. In a logic instruction condition, numeracy predicted probabilistic reasoning when GA was high, and GA affected nonprobabilistic reasoning directly. Thinking dispositions moderated neither relationship. Instead, instructions facilitated reasoning for low thinking disposition/high-ability participants, suggesting that logic instructions cued low thinking disposition individuals to engage in higher order reflective processing. The evidence is consistent with the proposals that reflective processes are essential to the allocation of algorithmic resources, and algorithmic resources are necessary for effective mindware implementation.

Adolescents' base rate judgments, metastrategic understanding, and stereotype endorsement

Developmental reversals are counterintuitive age trends wherein age is negatively related to optimal responding. We addressed the claims that reversals in judgments and decisions are unlikely between late childhood and adolescence. Children and adolescents indicated the extent to which they endorsed stereotypes salient to adolescents, responded to problems in which base rate evidence conflicted with evidence based on anecdotal evidence (i.e., anecdotal problems) or stereotype-relevant evidence (i.e., stereotypical problems), and indicated the basis for their responses. Normative responses increased with age on anecdotal problems and decreased with age on stereotypical problems, indicating a developmental reversal on the latter problem type. Metaprocedural competence mediated the age-related increases on anecdotal problems and moderated the age-related declines on stereotypical problems; furthermore, on stereotypical problems, endorsement of relevant stereotypes mediated the age-related declines in performance. Findings are discussed from a dual-process perspective that emphasizes gist-based problem representations, a "metacognitive gap," and age-related dissociations between stereotypical problems and metaprocedural competence.

Heuristics and biases: interactions among numeracy, ability, and reflectiveness predict normative responding

In Stanovich's (2009a, 2011) dual-process theory, analytic processing occurs in the algorithmic and reflective minds. Thinking dispositions, indexes of reflective mind functioning, are believed to regulate operations at the algorithmic level, indexed by general cognitive ability. General limitations at the algorithmic level impose constraints on, and affect the adequacy of, specific strategies and abilities (e.g., numeracy). In a study of 216 undergraduates, the hypothesis that thinking dispositions and general ability moderate the relationship between numeracy (understanding of mathematical concepts and attention to numerical information) and normative responses on probabilistic heuristics and biases (HB) problems was tested. Although all three individual difference measures predicted normative responses, the numeracy-normative response association depended on thinking dispositions and general ability. Specifically, numeracy directly affected normative responding only at relatively high levels of thinking dispositions and general ability. At low levels of thinking dispositions, neither general ability nor numeric skills related to normative responses. Discussion focuses on the consistency of these findings with the hypothesis that the implementation of specific skills is constrained by limitations at both the reflective level and the algorithmic level, methodological limitations that prohibit definitive conclusions, and alternative explanations.