1
|
Wang M, Nie QY. A computational account of conflict processing during mental imagery. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2024:10.3758/s13415-024-01201-z. [PMID: 39085587 DOI: 10.3758/s13415-024-01201-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 06/03/2024] [Indexed: 08/02/2024]
Abstract
Previous studies examining conflict processing within the context of a color-word Stroop task have focused on both stimulus and response conflicts. However, it has been unclear whether conflict can emerge independently of stimulus conflict. In this study, a novel arrow-gaze mental-rotation Stroop task was introduced to explore the interplay between conflict processing and mental rotation. A modelling approach was utilized to provide a process-level account of the findings. The results of our Stroop task indicate that conflict can emerge from mental rotation in the absence of stimulus conflict. The strength of this imagery conflict effect decreases and even reverses as mental rotation angles increase. Additionally, it was observed that participants responded more quickly and with greater accuracy to small rather than large face orientations. A comparison of three conflict diffusion models-the diffusion model for conflict tasks (DMC), the dual-stage two-phase model (DSTP), and the shrinking spotlight model (SSP)-yielded consistent support for the DSTP over the DMC and SSP in the majority of instances. The DSTP account of the experimental results revealed an increased nondecision time with increasing mental rotation, a reduction in interference from incompatible stimuli, and an improved drift rate in response selection phase, which suggests enhanced cognitive control. The findings from the model-based analysis provide evidence for a novel interaction between cognitive control and mental rotation.
Collapse
Affiliation(s)
- Mengxiao Wang
- Centre for Cognitive and Brain Sciences, University of Macau, Taipa, Macau
- Department of Electrical and Computer Engineering, University of Macau, Taipa, Macau
| | - Qi-Yang Nie
- Centre for Cognitive and Brain Sciences, University of Macau, Taipa, Macau.
- Department of Psychology, University of Macau, Taipa, Macau.
| |
Collapse
|
2
|
Heathcote A, Matzke D. Winner Takes All! What Are Race Models, and Why and How Should Psychologists Use Them? CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE 2022. [DOI: 10.1177/09637214221095852] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Interest in the processes that mediate between stimuli and responses is at the heart of most modern psychology and neuroscience. These processes cannot be directly measured but instead must be inferred from observed responses. Race models, through their ability to account for both response choices and response times, have been a key enabler of such inferences. Examples of such models appeared contemporaneously with the cognitive revolution, and since then have become increasingly prominent and elaborated, so that psychologists now have a powerful array of race models at their disposal. We showcase the state of the art for race models and describe why and how they are used.
Collapse
Affiliation(s)
- Andrew Heathcote
- School of Psychology, University of Newcastle
- Department of Psychology, University of Amsterdam
| | - Dora Matzke
- Department of Psychology, University of Amsterdam
| |
Collapse
|
3
|
Manning C, Hassall CD, Hunt LT, Norcia AM, Wagenmakers EJ, Evans NJ, Scerif G. Behavioural and neural indices of perceptual decision-making in autistic children during visual motion tasks. Sci Rep 2022; 12:6072. [PMID: 35414064 PMCID: PMC9005733 DOI: 10.1038/s41598-022-09885-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Accepted: 03/14/2022] [Indexed: 11/30/2022] Open
Abstract
Many studies report atypical responses to sensory information in autistic individuals, yet it is not clear which stages of processing are affected, with little consideration given to decision-making processes. We combined diffusion modelling with high-density EEG to identify which processing stages differ between 50 autistic and 50 typically developing children aged 6-14 years during two visual motion tasks. Our pre-registered hypotheses were that autistic children would show task-dependent differences in sensory evidence accumulation, alongside a more cautious decision-making style and longer non-decision time across tasks. We tested these hypotheses using hierarchical Bayesian diffusion models with a rigorous blind modelling approach, finding no conclusive evidence for our hypotheses. Using a data-driven method, we identified a response-locked centro-parietal component previously linked to the decision-making process. The build-up in this component did not consistently relate to evidence accumulation in autistic children. This suggests that the relationship between the EEG measure and diffusion-modelling is not straightforward in autistic children. Compared to a related study of children with dyslexia, motion processing differences appear less pronounced in autistic children. Exploratory analyses also suggest weak evidence that ADHD symptoms moderate perceptual decision-making in autistic children.
Collapse
Affiliation(s)
- Catherine Manning
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
- School of Psychology and Clinical Language Sciences, University of Reading, Reading, UK.
| | | | | | | | - Eric-Jan Wagenmakers
- Faculty of Social and Behavioural Sciences, University of Amsterdam, Amsterdam, The Netherlands
| | - Nathan J Evans
- School of Psychology, University of Queensland, Brisbane, Australia
| | - Gaia Scerif
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| |
Collapse
|
4
|
Weigard A, Sripada C. Task-general efficiency of evidence accumulation as a computationally-defined neurocognitive trait: Implications for clinical neuroscience. BIOLOGICAL PSYCHIATRY GLOBAL OPEN SCIENCE 2022; 1:5-15. [PMID: 35317408 DOI: 10.1016/j.bpsgos.2021.02.001] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023] Open
Abstract
Quantifying individual differences in higher-order cognitive functions is a foundational area of cognitive science that also has profound implications for research on psychopathology. For the last two decades, the dominant approach in these fields has been to attempt to fractionate higher-order functions into hypothesized components (e.g., "inhibition", "updating") through a combination of experimental manipulation and factor analysis. However, the putative constructs obtained through this paradigm have recently been met with substantial criticism on both theoretical and empirical grounds. Concurrently, an alternative approach has emerged focusing on parameters of formal computational models of cognition that have been developed in mathematical psychology. These models posit biologically plausible and experimentally validated explanations of the data-generating process for cognitive tasks, allowing them to be used to measure the latent mechanisms that underlie performance. One of the primary insights provided by recent applications of such models is that individual and clinical differences in performance on a wide variety of cognitive tasks, ranging from simple choice tasks to complex executive paradigms, are largely driven by efficiency of evidence accumulation (EEA), a computational mechanism defined by sequential sampling models. This review assembles evidence for the hypothesis that EEA is a central individual difference dimension that explains neurocognitive deficits in multiple clinical disorders and identifies ways in which in this insight can advance clinical neuroscience research. We propose that recognition of EEA as a major driver of neurocognitive differences will allow the field to make clearer inferences about cognitive abnormalities in psychopathology and their links to neurobiology.
Collapse
|
5
|
Manning C, Hassall CD, Hunt LT, Norcia AM, Wagenmakers EJ, Snowling MJ, Scerif G, Evans NJ. Visual Motion and Decision-Making in Dyslexia: Reduced Accumulation of Sensory Evidence and Related Neural Dynamics. J Neurosci 2022; 42:121-134. [PMID: 34782439 PMCID: PMC8741156 DOI: 10.1523/jneurosci.1232-21.2021] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2021] [Revised: 09/15/2021] [Accepted: 09/20/2021] [Indexed: 11/21/2022] Open
Abstract
Children with and without dyslexia differ in their behavioral responses to visual information, particularly when required to pool dynamic signals over space and time. Importantly, multiple processes contribute to behavioral responses. Here we investigated which processing stages are affected in children with dyslexia when performing visual motion processing tasks, by combining two methods that are sensitive to the dynamic processes leading to responses. We used a diffusion model which decomposes response time and accuracy into distinct cognitive constructs, and high-density EEG. Fifty children with dyslexia (24 male) and 50 typically developing children (28 male) 6-14 years of age judged the direction of motion as quickly and accurately as possible in two global motion tasks (motion coherence and direction integration), which varied in their requirements for noise exclusion. Following our preregistered analyses, we fitted hierarchical Bayesian diffusion models to the data, blinded to group membership. Unblinding revealed reduced evidence accumulation in children with dyslexia compared with typical children for both tasks. Additionally, we identified a response-locked EEG component which was maximal over centro-parietal electrodes which indicated a neural correlate of reduced drift rate in dyslexia in the motion coherence task, thereby linking brain and behavior. We suggest that children with dyslexia tend to be slower to extract sensory evidence from global motion displays, regardless of whether noise exclusion is required, thus furthering our understanding of atypical perceptual decision-making processes in dyslexia.SIGNIFICANCE STATEMENT Reduced sensitivity to visual information has been reported in dyslexia, with a lively debate about whether these differences causally contribute to reading difficulties. In this large preregistered study with a blind modeling approach, we combine state-of-the art methods in both computational modeling and EEG analysis to pinpoint the stages of processing that are atypical in children with dyslexia in two visual motion tasks that vary in their requirement for noise exclusion. We find reduced evidence accumulation in children with dyslexia across both tasks, and identify a neural marker, allowing us to link brain and behavior. We show that children with dyslexia exhibit general difficulties with extracting sensory evidence from global motion displays, not just in tasks that require noise exclusion.
Collapse
Affiliation(s)
- Catherine Manning
- Department of Experimental Psychology, University of Oxford, Oxford, Oxfordshire, United Kingdom, OX2 6GG
- School of Psychology and Clinical Language Sciences, University of Reading, Reading, Berkshire, United Kingdom, RG6 6ES
| | - Cameron D Hassall
- Department of Psychiatry, University of Oxford, Oxford, Oxfordshire, United Kingdom, OX3 7JX
| | - Laurence T Hunt
- Department of Psychiatry, University of Oxford, Oxford, Oxfordshire, United Kingdom, OX3 7JX
| | - Anthony M Norcia
- Department of Psychology, Stanford University, Stanford, CA 94305, US
| | - Eric-Jan Wagenmakers
- Faculty of Social and Behavioural Sciences, University of Amsterdam, 1001 NH Amsterdam, The Netherlands
| | - Margaret J Snowling
- Department of Experimental Psychology, University of Oxford, Oxford, Oxfordshire, United Kingdom, OX2 6GG
| | - Gaia Scerif
- Department of Experimental Psychology, University of Oxford, Oxford, Oxfordshire, United Kingdom, OX2 6GG
| | - Nathan J Evans
- School of Psychology, University of Queensland, Brisbane, QLD 4072 Australia
| |
Collapse
|
6
|
Abstract
The effects of distraction on responses manifest in three ways: prolonged reaction times, and increased error and response omission rates. However, the latter effect is often ignored or assumed to be due to a separate cognitive process. We investigated omissions occurring in two paradigms that manipulated distraction. One required simple stimulus detection of younger participants, the second required choice responses and was completed by both younger and older participants. We fit data from these paradigms with a model that identifies three causes of omissions: two are related to the process of accumulating the evidence on which a response is based: intrinsic omissions (due to between-trial variation in accumulation rates making it impossible to ever reach the evidence threshold) and design omissions (due to response windows that cause slow responses not to be recorded; a third, contaminant omissions, allows for a cause unrelated to the response process. In both data sets systematic differences in omission rates across conditions were accounted for by task-related omissions. Intrinsic omissions played a lesser role than design omissions, even though the presence of design omissions was not evident in descriptive analyses of the data. The model provided an accurate account of all aspects of the detection data and the choice-response data, but slightly underestimated overall omissions in the choice paradigm, particularly in older participants, suggesting that further investigation of contaminant omission effects is needed.
Collapse
|
7
|
Crüwell S, Evans NJ. Preregistration in diverse contexts: a preregistration template for the application of cognitive models. ROYAL SOCIETY OPEN SCIENCE 2021; 8:210155. [PMID: 34659776 PMCID: PMC8511762 DOI: 10.1098/rsos.210155] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/29/2021] [Accepted: 08/16/2021] [Indexed: 05/20/2023]
Abstract
In recent years, open science practices have become increasingly popular in psychology and related sciences. These practices aim to increase rigour and transparency in science as a potential response to the challenges posed by the replication crisis. Many of these reforms-including the increasingly used preregistration-have been designed for purely experimental work that tests straightforward hypotheses with standard inferential statistical analyses, such as assessing whether an experimental manipulation has an effect on a variable of interest. But psychology is a diverse field of research. The somewhat narrow focus of the prevalent discussions surrounding and templates for preregistration has led to debates on how appropriate these reforms are for areas of research with more diverse hypotheses and more intricate methods of analysis, such as cognitive modelling research within mathematical psychology. Our article attempts to bridge the gap between open science and mathematical psychology, focusing on the type of cognitive modelling that Crüwell et al. (Crüwell S, Stefan AM, Evans NJ. 2019 Robust standards in cognitive science. Comput. Brain Behav. 2, 255-265) labelled model application, where researchers apply a cognitive model as a measurement tool to test hypotheses about parameters of the cognitive model. Specifically, we (i) discuss several potential researcher degrees of freedom within model application, (ii) provide the first preregistration template for model application and (iii) provide an example of a preregistered model application using our preregistration template. More broadly, we hope that our discussions and concrete proposals constructively advance the mostly abstract current debate surrounding preregistration in cognitive modelling, and provide a guide for how preregistration templates may be developed in other diverse or intricate research contexts.
Collapse
Affiliation(s)
- Sophia Crüwell
- Meta-Research Innovation Center Berlin (METRIC-B), QUEST Center for Transforming Biomedical Research, Berlin Institute of Health, Charité – Universitätsmedizin Berlin, Berlin, Germany
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
- Department of History and Philosophy of Science, University of Cambridge, Cambridge, UK
| | - Nathan J. Evans
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
- School of Psychology, University of Queensland, Queensland, Australia
- School of Psychology, University of Newcastle, Callaghan, Australia
| |
Collapse
|
8
|
Weigard AS, Brislin SJ, Cope LM, Hardee JE, Martz ME, Ly A, Zucker RA, Sripada C, Heitzeg MM. Evidence accumulation and associated error-related brain activity as computationally-informed prospective predictors of substance use in emerging adulthood. Psychopharmacology (Berl) 2021; 238:2629-2644. [PMID: 34173032 PMCID: PMC8452274 DOI: 10.1007/s00213-021-05885-w] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/07/2020] [Accepted: 05/27/2021] [Indexed: 01/05/2023]
Abstract
RATIONALE Substance use peaks during the developmental period known as emerging adulthood (ages 18-25), but not every individual who uses substances during this period engages in frequent or problematic use. Although individual differences in neurocognition appear to predict use severity, mechanistic neurocognitive risk factors with clear links to both behavior and neural circuitry have yet to be identified. Here, we aim to do so with an approach rooted in computational psychiatry, an emerging field in which formal models are used to identify candidate biobehavioral dimensions that confer risk for psychopathology. OBJECTIVES We test whether lower efficiency of evidence accumulation (EEA), a computationally characterized individual difference variable that drives performance on the go/no-go and other neurocognitive tasks, is a risk factor for substance use in emerging adults. METHODS AND RESULTS In an fMRI substudy within a sociobehavioral longitudinal study (n = 106), we find that lower EEA and reductions in a robust neural-level correlate of EEA (error-related activations in salience network structures) measured at ages 18-21 are both prospectively related to greater substance use during ages 22-26, even after adjusting for other well-known risk factors. Results from Bayesian model comparisons corroborated inferences from conventional hypothesis testing and provided evidence that both EEA and its neuroimaging correlates contain unique predictive information about substance use involvement. CONCLUSIONS These findings highlight EEA as a computationally characterized neurocognitive risk factor for substance use during a critical developmental period, with clear links to both neuroimaging measures and well-established formal theories of brain function.
Collapse
Affiliation(s)
- Alexander S Weigard
- Department of Psychiatry, University of Michigan, Rachel Upjohn Building, 4250 Plymouth Road, Ann Arbor, MI, 48109, USA.
| | - Sarah J Brislin
- Department of Psychiatry, University of Michigan, Rachel Upjohn Building, 4250 Plymouth Road, Ann Arbor, MI, 48109, USA
| | - Lora M Cope
- Department of Psychiatry, University of Michigan, Rachel Upjohn Building, 4250 Plymouth Road, Ann Arbor, MI, 48109, USA
| | - Jillian E Hardee
- Department of Psychiatry, University of Michigan, Rachel Upjohn Building, 4250 Plymouth Road, Ann Arbor, MI, 48109, USA
| | - Meghan E Martz
- Department of Psychiatry, University of Michigan, Rachel Upjohn Building, 4250 Plymouth Road, Ann Arbor, MI, 48109, USA
| | - Alexander Ly
- Department of Psychological Methods, University of Amsterdam, Amsterdam, The Netherlands
- Machine Learning Group, Centrum Wiskunde & Informatica, Amsterdam, The Netherlands
| | - Robert A Zucker
- Department of Psychiatry, University of Michigan, Rachel Upjohn Building, 4250 Plymouth Road, Ann Arbor, MI, 48109, USA
| | - Chandra Sripada
- Department of Psychiatry, University of Michigan, Rachel Upjohn Building, 4250 Plymouth Road, Ann Arbor, MI, 48109, USA
| | - Mary M Heitzeg
- Department of Psychiatry, University of Michigan, Rachel Upjohn Building, 4250 Plymouth Road, Ann Arbor, MI, 48109, USA
| |
Collapse
|
9
|
Think fast! The implications of emphasizing urgency in decision-making. Cognition 2021; 214:104704. [PMID: 33975126 DOI: 10.1016/j.cognition.2021.104704] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2020] [Revised: 03/10/2021] [Accepted: 03/22/2021] [Indexed: 11/23/2022]
Abstract
Evidence accumulation models (EAMs) have become the dominant explanation of how the decision-making process operates, proposing that decisions are the result of a process of evidence accumulation. The primary use of EAMs has been as "measurement tools" of the underlying decision-making process, where researchers apply EAMs to empirical data to estimate participants' task ability (i.e., the "drift rate"), response caution (i.e., the "decision threshold"), and the time taken for other processes (i.e., the "non-decision time"), making EAMs a powerful tool for discriminating between competing psychological theories. Recent studies have brought into question the mapping between the latent parameters of EAMs and the theoretical constructs that they are thought to represent, showing that emphasizing urgent responding - which intuitively should selectively influence decision threshold - may also influence drift rate and/or non-decision time. However, these findings have been mixed, leading to differences in opinion between experts in the field. The current study aims to provide a more conclusive answer to the implications of emphasizing urgent responding, providing a re-analysis of 6 data sets from previous studies using two different EAMs - the diffusion model and the linear ballistic accumulator (LBA) - with state-of-the-art methods for model selection based inference. The findings display clear evidence for a difference in conclusions between the two models, with the diffusion model suggesting that decision threshold and non-decision time decrease when urgency is emphasized, and the LBA suggesting that decision threshold and drift rate decrease when urgency is emphasized. Furthermore, although these models disagree regarding whether non-decision time or drift rate decrease under urgency emphasis, both show clear evidence that emphasizing urgency does not selectively influence decision threshold. These findings suggest that researchers should revise their assumptions about certain experimental manipulations, the specification of certain EAMs, or perhaps both.
Collapse
|
10
|
|
11
|
|
12
|
Ballard T, Palada H, Griffin M, Neal A. An Integrated Approach to Testing Dynamic, Multilevel Theory: Using Computational Models to Connect Theory, Model, and Data. ORGANIZATIONAL RESEARCH METHODS 2019. [DOI: 10.1177/1094428119881209] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Some of the most influential theories in organizational sciences explicitly describe a dynamic, multilevel process. Yet the inherent complexity of such theories makes them difficult to test. These theories often describe multiple subprocesses that interact reciprocally over time at different levels of analysis and over different time scales. Computational (i.e., mathematical) modeling is increasingly advocated as a method for developing and testing theories of this type. In organizational sciences, however, efforts that have been made to test models empirically are often indirect. We argue that the full potential of computational modeling as a tool for testing dynamic, multilevel theory is yet to be realized. In this article, we demonstrate an approach to testing dynamic, multilevel theory using computational modeling. The approach uses simulations to generate model predictions and Bayesian parameter estimation to fit models to empirical data and facilitate model comparisons. This approach enables a direct integration between theory, model, and data that we believe enables a more rigorous test of theory.
Collapse
Affiliation(s)
- Timothy Ballard
- School of Psychology, University of Queensland, St Lucia, Australia
| | - Hector Palada
- School of Psychology, University of Queensland, St Lucia, Australia
| | - Mark Griffin
- Future of Work Institute, Curtin University, Perth, WA, Australia
| | - Andrew Neal
- School of Psychology, University of Queensland, St Lucia, Australia
| |
Collapse
|
13
|
Abstract
Evidence accumulation models (EAMs) have become the dominant models of rapid decision-making. Several variants of these models have been proposed, ranging from the simple linear ballistic accumulator (LBA) to the more complex leaky-competing accumulator (LCA), and further extensions that include time-varying rates of evidence accumulation or decision thresholds. Although applications of the simpler variants have been widespread, applications of the more complex models have been fewer, largely due to their intractable likelihood function and the computational cost of mass simulation. Here, I present a framework for efficiently fitting complex EAMs, which uses a new, efficient method of simulating these models. I find that the majority of simulation time is taken up by random number generation (RNG) from the normal distribution, needed for the stochastic noise of the differential equation. To reduce this inefficiency, I propose using the well-known concept within computer science of "look-up tables" (LUTs) as an approximation to the inverse cumulative density function (iCDF) method of RNG, which I call "LUT-iCDF". I show that when using an appropriately sized LUT, simulations using LUT-iCDF closely match those from the standard RNG method in R. My framework, which I provide a detailed tutorial on how to implement, includes C code for 12 different variants of EAMs using the LUT-iCDF method, and should make the implementation of complex EAMs easier and faster.
Collapse
Affiliation(s)
- Nathan J Evans
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands.
- Department of Psychology, Vanderbilt University, Nashville, TN, USA.
| |
Collapse
|
14
|
|
15
|
Evans NJ. Assessing the practical differences between model selection methods in inferences about choice response time tasks. Psychon Bull Rev 2019; 26:1070-1098. [PMID: 30783896 PMCID: PMC6710222 DOI: 10.3758/s13423-018-01563-9] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Evidence accumulations models (EAMs) have become the dominant modeling framework within rapid decision-making, using choice response time distributions to make inferences about the underlying decision process. These models are often applied to empirical data as "measurement tools", with different theoretical accounts being contrasted within the framework of the model. Some method is then needed to decide between these competing theoretical accounts, as only assessing the models on their ability to fit trends in the empirical data ignores model flexibility, and therefore, creates a bias towards more flexible models. However, there is no objectively optimal method to select between models, with methods varying in both their computational tractability and theoretical basis. I provide a systematic comparison between nine different model selection methods using a popular EAM-the linear ballistic accumulator (LBA; Brown & Heathcote, Cognitive Psychology 57(3), 153-178 2008)-in a large-scale simulation study and the empirical data of Dutilh et al. (Psychonomic Bulletin and Review, 1-19 2018). I find that the "predictive accuracy" class of methods (i.e., the Akaike Information Criterion [AIC], the Deviance Information Criterion [DIC], and the Widely Applicable Information Criterion [WAIC]) make different inferences to the "Bayes factor" class of methods (i.e., the Bayesian Information Criterion [BIC], and Bayes factors) in many, but not all, instances, and that the simpler methods (i.e., AIC and BIC) make inferences that are highly consistent with their more complex counterparts. These findings suggest that researchers should be able to use simpler "parameter counting" methods when applying the LBA and be confident in their inferences, but that researchers need to carefully consider and justify the general class of model selection method that they use, as different classes of methods often result in different inferences.
Collapse
Affiliation(s)
- Nathan J Evans
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands.
| |
Collapse
|
16
|
Evans NJ, Annis J. Thermodynamic integration via differential evolution: A method for estimating marginal likelihoods. Behav Res Methods 2019; 51:930-947. [PMID: 30604038 PMCID: PMC6478771 DOI: 10.3758/s13428-018-1172-y] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
A typical goal in cognitive psychology is to select the model that provides the best explanation of the observed behavioral data. The Bayes factor provides a principled approach for making these selections, though the integral required to calculate the marginal likelihood for each model is intractable for most cognitive models. In these cases, Monte Carlo techniques must be used to approximate the marginal likelihood, such as thermodynamic integration (TI; Friel & Pettitt, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(3), 589-607 2008; Lartillot & Philippe, Systematic Biology, 55(2), 195-207 2006), which relies on sampling from the posterior at different powers (called power posteriors). TI can become computationally expensive when using population Markov chain Monte Carlo (MCMC) approaches such as differential evolution MCMC (DE-MCMC; Turner et al., Psychological Methods, 18(3), 368 2013) that require several interacting chains per power posterior. Here, we propose a method called thermodynamic integration via differential evolution (TIDE), which aims to reduce the computational burden associated with TI by using a single chain per power posterior (R code available at https://osf.io/ntmgw/ ). We show that when applied to non-hierarchical models, TIDE produces an approximation of the marginal likelihood that closely matches TI. When extended to hierarchical models, we find that certain assumptions about the dependence between the individual- and group-level parameters samples (i.e., dependent/independent) have sizable effects on the TI approximated marginal likelihood. We propose two possible extensions of TIDE to hierarchical models, which closely match the marginal likelihoods obtained through TI with dependent/independent sampling in many, but not all, situations. Based on these findings, we believe that TIDE provides a promising method for estimating marginal likelihoods, though future research should focus on a detailed comparison between the methods of estimating marginal likelihoods for cognitive models.
Collapse
Affiliation(s)
- Nathan J. Evans
- Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
| | - Jeffrey Annis
- Department of Psychology, Vanderbilt University, Nashville, TN 37235 USA
| |
Collapse
|
17
|
Knowles JP, Evans NJ, Burke D. Some Evidence for an Association Between Early Life Adversity and Decision Urgency. Front Psychol 2019; 10:243. [PMID: 30804859 PMCID: PMC6377396 DOI: 10.3389/fpsyg.2019.00243] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2018] [Accepted: 01/24/2019] [Indexed: 11/24/2022] Open
Abstract
The relationship between early life adversity and adult outcomes is traditionally investigated relative to risk and protective factors (e.g., resilience, cognitive appraisal), and poor self-control or decision-making. However, life history theory suggests this relationship may be adaptive-underpinned by mechanisms that use early environmental cues to alter the developmental trajectory toward more short-term strategies. These short-term strategies have some theoretical overlap with the most common process models of decision-making-evidence accumulation models-which model decision urgency as a decision threshold. The current study examined the relationship between decision urgency (through the linear ballistic accumulator) and early life adversity. A mixture of analysis methods, including a joint model analysis designed to explicitly account for uncertainty in estimated decision urgency values, revealed weak-to-strong evidence in favor of a relationship between decision urgency and early life adversity, suggesting a possible effect of life history strategy on even the most basic decisions.
Collapse
Affiliation(s)
- Johanne P. Knowles
- School of Psychology, University of Newcastle, Callaghan, NSW, Australia
| | - Nathan J. Evans
- Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
| | - Darren Burke
- School of Psychology, University of Newcastle, Callaghan, NSW, Australia
| |
Collapse
|
18
|
Response-time data provide critical constraints on dynamic models of multi-alternative, multi-attribute choice. Psychon Bull Rev 2019; 26:901-933. [DOI: 10.3758/s13423-018-1557-z] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|