Optimizing sequential decisions in the drift-diffusion model

Trial-history biases in evidence accumulation can give rise to apparent lapses in decision-making

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, previous work has suggested that they covary in their prevalence and that their proposed neural substrates overlap. Here we demonstrate that during decision-making, history biases and apparent lapses can both arise from a common cognitive process that is optimal under mistaken beliefs that the world is changing i.e. nonstationary. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct decision-making datasets of male rats, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Trial-history biases in evidence accumulation can give rise to apparent lapses

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, several hints in the literature suggest that they covary in their prevalence and that their proposed neural substrates overlap - what could underlie these links? Here we demonstrate that history biases and apparent lapses can both arise from a common cognitive process that is normative under misbeliefs about non-stationarity in the world. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct rat decision-making datasets, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Informational Entropy Threshold as a Physical Mechanism for Explaining Tree-like Decision Making in Humans

While approaches based on physical grounds (such as the drift-diffusion model-DDM) have been exhaustively used in psychology and neuroscience to describe perceptual decision making in humans, similar approaches to complex situations, such as sequential (tree-like) decisions, are still scarce. For such scenarios that involve a reflective prospection of future options, we offer a plausible mechanism based on the idea that subjects can carry out an internal computation of the uncertainty about the different options available, which is computed through the corresponding Shannon entropy. When the amount of information gathered through sensory evidence is enough to reach a given threshold in the entropy, this will trigger the decision. Experimental evidence in favor of this entropy-based mechanism was provided by exploring human performance during navigation through a maze on a computer screen monitored with the help of eye trackers. In particular, our analysis allows us to prove that (i) prospection is effectively used by humans during such navigation tasks, and an indirect quantification of the level of prospection used is attainable; in addition, (ii) the distribution of decision times during the task exhibits power-law tails, a feature that our entropy-based mechanism is able to explain, unlike traditional (DDM-like) frameworks.

Suboptimal human inference can invert the bias-variance trade-off for decisions with asymmetric evidence

Solutions to challenging inference problems are often subject to a fundamental trade-off between: 1) bias (being systematically wrong) that is minimized with complex inference strategies, and 2) variance (being oversensitive to uncertain observations) that is minimized with simple inference strategies. However, this trade-off is based on the assumption that the strategies being considered are optimal for their given complexity and thus has unclear relevance to forms of inference based on suboptimal strategies. We examined inference problems applied to rare, asymmetrically available evidence, which a large population of human subjects solved using a diverse set of strategies that varied in form and complexity. In general, subjects using more complex strategies tended to have lower bias and variance, but with a dependence on the form of strategy that reflected an inversion of the classic bias-variance trade-off: subjects who used more complex, but imperfect, Bayesian-like strategies tended to have lower variance but higher bias because of incorrect tuning to latent task features, whereas subjects who used simpler heuristic strategies tended to have higher variance because they operated more directly on the observed samples but lower, near-normative bias. Our results help define new principles that govern individual differences in behavior that depends on rare-event inference and, more generally, about the information-processing trade-offs that can be sensitive to not just the complexity, but also the optimality, of the inference process.

People use diverse strategies to make inferences about the world around them, often based on limited evidence. Such inference strategies may be simple but prone to systematic errors or more complex and accurate, but such trends need not always be the rule. We modeled and measured how human participants made rare-event decisions in a preregistered, online study. The participants tended to use suboptimal decision strategies that reflected an inversion of the classic bias-variance trade-off: some used complex, nearly normative strategies with mistuned evidence weights that corresponded to relatively high choice biases but lower choice variance, whereas others used simpler heuristic strategies that corresponded to lower biases but higher variance. These relationships illustrate structure in suboptimality that can be used to identify systematic sources of human errors.

Is value-based choice repetition susceptible to medial frontal transcranial direct current stimulation (tDCS)? A preregistered study

In value-based decision making, people have to weigh different options based on their subjective value. This process, however, also is influenced by choice biases, such as choice repetition: in a series of choices, people are more likely to repeat their decision than to switch to a different choice. Previously, it was shown that transcranial direct current stimulation (tDCS) can affect such choice biases. We applied tDCS over the medial prefrontal cortex to investigate whether tDCS can alter choice repetition in value-based decision making. In a preregistered study, we applied anodal, cathodal, and sham tDCS stimulation to 52 participants. While we found robust choice repetition effects, we did not find support for an effect of tDCS stimulation. We discuss these findings within the larger scope of the tDCS literature and highlight the potential roles of interindividual variability and current density strength.

A flexible framework for simulating and fitting generalized drift-diffusion models

The drift-diffusion model (DDM) is an important decision-making model in cognitive neuroscience. However, innovations in model form have been limited by methodological challenges. Here, we introduce the generalized drift-diffusion model (GDDM) framework for building and fitting DDM extensions, and provide a software package which implements the framework. The GDDM framework augments traditional DDM parameters through arbitrary user-defined functions. Models are solved numerically by directly solving the Fokker-Planck equation using efficient numerical methods, yielding a 100-fold or greater speedup over standard methodology. This speed allows GDDMs to be fit to data using maximum likelihood on the full response time (RT) distribution. We demonstrate fitting of GDDMs within our framework to both animal and human datasets from perceptual decision-making tasks, with better accuracy and fewer parameters than several DDMs implemented using the latest methodology, to test hypothesized decision-making mechanisms. Overall, our framework will allow for decision-making model innovation and novel experimental designs.

Optimal models of decision-making in dynamic environments

Nature is in constant flux, so animals must account for changes in their environment when making decisions. How animals learn the timescale of such changes and adapt their decision strategies accordingly is not well understood. Recent psychophysical experiments have shown humans and other animals can achieve near-optimal performance at two alternative forced choice (2AFC) tasks in dynamically changing environments. Characterization of performance requires the derivation and analysis of computational models of optimal decision-making policies on such tasks. We review recent theoretical work in this area, and discuss how models compare with subjects' behavior in tasks where the correct choice or evidence quality changes in dynamic, but predictable, ways.

Drift-diffusion models for multiple-alternative forced-choice decision making

The canonical computational model for the cognitive process underlying two-alternative forced-choice decision making is the so-called drift-diffusion model (DDM). In this model, a decision variable keeps track of the integrated difference in sensory evidence for two competing alternatives. Here I extend the notion of a drift-diffusion process to multiple alternatives. The competition between n alternatives takes place in a linear subspace of [Formula: see text] dimensions; that is, there are [Formula: see text] decision variables, which are coupled through correlated noise sources. I derive the multiple-alternative DDM starting from a system of coupled, linear firing rate equations. I also show that a Bayesian sequential probability ratio test for multiple alternatives is, in fact, equivalent to these same linear DDMs, but with time-varying thresholds. If the original neuronal system is nonlinear, one can once again derive a model describing a lower-dimensional diffusion process. The dynamics of the nonlinear DDM can be recast as the motion of a particle on a potential, the general form of which is given analytically for an arbitrary number of alternatives.