Predicting the Past, Remembering the Future

Scanning a compressed ordered representation of the future

Several authors have suggested a deep symmetry between the psychological processes that underlie our ability to remember the past and make predictions about the future. The judgment of recency (JOR) task measures temporal order judgments for the past by presenting pairs of probe stimuli; participants choose the probe that was presented more recently. We performed a short-term relative JOR task and introduced a novel judgment of imminence (JOI) task to study temporal order judgments for the future. In the JOR task, participants were presented with a sequence of stimuli and asked to choose which of two probe stimuli was presented closer to the present. In the JOI task, participants were trained on a probabilistic sequence. After training, the sequence was interrupted with probe stimuli. Participants were asked to choose which of two probe stimuli was expected to be presented closer to the present. Replicating prior work on JOR, we found that RT results supported a backward self-terminating search model operating on a temporally organized representation of the past. We also showed that RT distributions are consistent with this model and that the temporally organized representation is compressed. Critically, results for the JOI task probing expectations of the future suggest a forward self-terminating search model operating on a temporally organized representation of the future. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The power of negative and positive episodic memories

The power of episodic memories is that they bring a past moment into the present, providing opportunities for us to recall details of the experiences, reframe or update the memory, and use the retrieved information to guide our decisions. In these regards, negative and positive memories can be especially powerful: Life’s highs and lows are disproportionately represented in memory, and when they are retrieved, they often impact our current mood and thoughts and influence various forms of behavior. Research rooted in neuroscience and cognitive psychology has historically focused on memory for negative emotional content. Yet the study of autobiographical memories has highlighted the importance of positive emotional memories, and more recently, cognitive neuroscience methods have begun to clarify why positive memories may show powerful relations to mental wellbeing. Here, we review the models that have been proposed to explain why emotional memories are long-lasting (durable) and likely to be retrieved (accessible), describing how in overlapping—but distinctly separable—ways, positive and negative memories can be easier to retrieve, and more likely to influence behavior. We end by identifying potential implications of this literature for broader topics related to mental wellbeing, education, and workplace environments.

Oversampled and undersolved: Depressive rumination from an active inference perspective

Rumination is a widely recognized cognitive deviation in depression. Despite the recognition, researchers have struggled to explain why patients cannot disengage from the process, although it depresses their mood and fails to lead to effective problem-solving. We rethink rumination as repetitive but unsuccessful problem-solving attempts. Appealing to an active inference account, we suggest that adaptive problem-solving is based on the generation, evaluation, and performance of candidate policies that increase an organism's knowledge of its environment. We argue that the problem-solving process is distorted during rumination. Specifically, rumination is understood as engaging in excessive yet unsuccessful oversampling of policy candidates that do not resolve uncertainty. Because candidates are sampled from policies that were selected in states resembling one's current state, "bad" starting points (e.g., depressed mood, physical inactivity) make the problem-solving process vulnerable for generating a ruminative "halting problem". This problem leads to high opportunity costs, learned helplessness and diminished overt behavior. Besides reviewing evidence for the conceptual paths of this model, we discuss its neurophysiological correlates and point towards clinical implications.

Emergence of associative learning in a neuromorphic inference network

OBJECTIVE

In the theoretical framework of predictive coding and active inference, the brain can be viewed as instantiating a rich generative model of the world that predicts incoming sensory data while continuously updating its parameters via minimization of prediction errors. While this theory has been successfully applied to cognitive processes - by modelling the activity of functional neural networks at a mesoscopic scale - the validity of the approach when modelling neurons as an ensemble of inferring agents, in a biologically plausible architecture, remained to be explored.

APPROACH

We modelled a simplified cerebellar circuit with individual neurons acting as Bayesian agents to simulate the classical delayed eyeblink conditioning protocol. Neurons and synapses adjusted their activity to minimize their prediction error, which was used as the network cost function. This cerebellar network was then implemented in hardware by replicating digital neuronal elements via a low-power microcontroller.

MAIN RESULTS

Persistent changes of synaptic strength - that mirrored neurophysiological observations - emerged via local (neurocentric) prediction error minimization, leading to the expression of associative learning. The same paradigm was effectively emulated in low-power hardware showing remarkably efficient performance compared to conventional neuromorphic architectures.

SIGNIFICANCE

These findings show that: i) an ensemble of free energy minimizing neurons - organized in a biological plausible architecture - can recapitulate functional self-organization observed in nature, such as associative plasticity, and ii) a neuromorphic network of inference units can learn unsupervised tasks without embedding predefined learning rules in the circuit, thus providing a potential avenue to a novel form of brain-inspired artificial intelligence.

The Neural Representation of Events Is Dominated by Elements that Are Most Reliably Present

An episodic memory is specific to an event that occurred at a particular time and place. However, the elements that comprise the event-the location, the people present, and their actions and goals-might be shared with numerous other similar events. Does the brain preferentially represent certain elements of a remembered event? If so, which elements dominate its neural representation: those that are shared across similar events, or the novel elements that define a specific event? We addressed these questions by using a novel experimental paradigm combined with fMRI. Multiple events were created involving conversations between two individuals using the format of a television chat show. Chat show "hosts" occurred repeatedly across multiple events, whereas the "guests" were unique to only one event. Before learning the conversations, participants were scanned while viewing images or names of the (famous) individuals to be used in the study to obtain person-specific activity patterns. After learning all the conversations over a week, participants were scanned for a second time while they recalled each event multiple times. We found that during recall, person-specific activity patterns within the posterior midline network were reinstated for the hosts of the shows but not the guests, and that reinstatement of the hosts was significantly stronger than the reinstatement of the guests. These findings demonstrate that it is the more generic, familiar, and predictable elements of an event that dominate its neural representation compared with the more idiosyncratic, event-defining, elements.

The spatiotemporal organization of experience dictates hippocampal involvement in primary visual cortical plasticity

The hippocampus and neocortex are theorized to be crucial partners in the formation of long-term memories. Here, we assess hippocampal involvement in two related forms of experience-dependent plasticity in the primary visual cortex (V1) of mice. Like control animals, those with hippocampal lesions exhibit potentiation of visually evoked potentials after passive daily exposure to a phase-reversing oriented grating stimulus, which is accompanied by long-term habituation of a reflexive behavioral response. Thus, low-level recognition memory is formed independently of the hippocampus. However, response potentiation resulting from daily exposure to a fixed sequence of four oriented gratings is severely impaired in mice with hippocampal damage. A feature of sequence plasticity in V1 of controls, which is absent in lesioned mice, is the generation of predictive responses to an anticipated stimulus element when it is withheld or delayed. Thus, the hippocampus is involved in encoding temporally structured experience, even within the primary sensory cortex.

Sophisticated Inference

Active inference offers a first principle account of sentient behavior, from which special and important cases-for example, reinforcement learning, active learning, Bayes optimal inference, Bayes optimal design-can be derived. Active inference finesses the exploitation-exploration dilemma in relation to prior preferences by placing information gain on the same footing as reward or value. In brief, active inference replaces value functions with functionals of (Bayesian) beliefs, in the form of an expected (variational) free energy. In this letter, we consider a sophisticated kind of active inference using a recursive form of expected free energy. Sophistication describes the degree to which an agent has beliefs about beliefs. We consider agents with beliefs about the counterfactual consequences of action for states of affairs and beliefs about those latent states. In other words, we move from simply considering beliefs about "what would happen if I did that" to "what I would believe about what would happen if I did that." The recursive form of the free energy functional effectively implements a deep tree search over actions and outcomes in the future. Crucially, this search is over sequences of belief states as opposed to states per se. We illustrate the competence of this scheme using numerical simulations of deep decision problems.

Visual object recognition is facilitated by temporal community structure

Humans and others primates are highly attuned to temporal consistencies and regularities in their sensory environment and learn to predict such statistical structure. Moreover, in several instances, the presence of temporal structure has been found to facilitate procedural learning and to improve task performance. Here we extend these findings to visual object recognition and to presentation sequences in which mutually predictive objects form distinct clusters or "communities." Our results show that temporal community structure accelerates recognition learning and affects the order in which objects are learned ("onset of familiarity").

Generative models, linguistic communication and active inference

This paper presents a biologically plausible generative model and inference scheme that is capable of simulating communication between synthetic subjects who talk to each other. Building on active inference formulations of dyadic interactions, we simulate linguistic exchange to explore generative models that support dialogues. These models employ high-order interactions among abstract (discrete) states in deep (hierarchical) models. The sequential nature of language processing mandates generative models with a particular factorial structure-necessary to accommodate the rich combinatorics of language. We illustrate linguistic communication by simulating a synthetic subject who can play the 'Twenty Questions' game. In this game, synthetic subjects take the role of the questioner or answerer, using the same generative model. This simulation setup is used to illustrate some key architectural points and demonstrate that many behavioural and neurophysiological correlates of linguistic communication emerge under variational (marginal) message passing, given the right kind of generative model. For example, we show that theta-gamma coupling is an emergent property of belief updating, when listening to another.

Degeneracy and Redundancy in Active Inference

The notions of degeneracy and redundancy are important constructs in many areas, ranging from genomics through to network science. Degeneracy finds a powerful role in neuroscience, explaining key aspects of distributed processing and structure-function relationships in the brain. For example, degeneracy accounts for the superadditive effect of lesions on functional deficits in terms of a "many-to-one" structure-function mapping. In this paper, we offer a principled account of degeneracy and redundancy, when function is operationalized in terms of active inference, namely, a formulation of perception and action as belief updating under generative models of the world. In brief, "degeneracy" is quantified by the "entropy" of posterior beliefs about the causes of sensations, while "redundancy" is the "complexity" cost incurred by forming those beliefs. From this perspective, degeneracy and redundancy are complementary: Active inference tries to minimize redundancy while maintaining degeneracy. This formulation is substantiated using statistical and mathematical notions of degenerate mappings and statistical efficiency. We then illustrate changes in degeneracy and redundancy during the learning of a word repetition task. Finally, we characterize the effects of lesions-to intrinsic and extrinsic connections-using in silico disconnections. These numerical analyses highlight the fundamental difference between degeneracy and redundancy-and how they score distinct imperatives for perceptual inference and structure learning that are relevant to synthetic and biological intelligence.

Optimal forgetting: Semantic compression of episodic memories

It has extensively been documented that human memory exhibits a wide range of systematic distortions, which have been associated with resource constraints. Resource constraints on memory can be formalised in the normative framework of lossy compression, however traditional lossy compression algorithms result in qualitatively different distortions to those found in experiments with humans. We argue that the form of distortions is characteristic of relying on a generative model adapted to the environment for compression. We show that this semantic compression framework can provide a unifying explanation of a wide variety of memory phenomena. We harness recent advances in learning deep generative models, that yield powerful tools to approximate generative models of complex data. We use three datasets, chess games, natural text, and hand-drawn sketches, to demonstrate the effects of semantic compression on memory performance. Our model accounts for memory distortions related to domain expertise, gist-based distortions, contextual effects, and delayed recall.

Prediction and memory: A predictive coding account

The hippocampus is crucial for episodic memory, but it is also involved in online prediction. Evidence suggests that a unitary hippocampal code underlies both episodic memory and predictive processing, yet within a predictive coding framework the hippocampal-neocortical interactions that accompany these two phenomena are distinct and opposing. Namely, during episodic recall, the hippocampus is thought to exert an excitatory influence on the neocortex, to reinstate activity patterns across cortical circuits. This contrasts with empirical and theoretical work on predictive processing, where descending predictions suppress prediction errors to 'explain away' ascending inputs via cortical inhibition. In this hypothesis piece, we attempt to dissolve this previously overlooked dialectic. We consider how the hippocampus may facilitate both prediction and memory, respectively, by inhibiting neocortical prediction errors or increasing their gain. We propose that these distinct processing modes depend upon the neuromodulatory gain (or precision) ascribed to prediction error units. Within this framework, memory recall is cast as arising from fictive prediction errors that furnish training signals to optimise generative models of the world, in the absence of sensory data.

Predictive processing: Is the future just a memory?

Predictive processing seems like a radical departure from traditional theories of information processing in the brain, but a broader view of predictions highlights many similarities with standard frameworks. Predictive processing is memory and competitive bias in a new outlook-and we should use this correspondence to advance research on both fronts.

Deliberating trade-offs with the future

Many fundamental choices in life are intertemporal: they involve trade-offs between sooner and later outcomes. In recent years there has been a surge of interest into how people make intertemporal decisions, given that such decisions are ubiquitous in everyday life and central in domains from substance use to climate change action. While it is clear that people make decisions according to rules, intuitions and habits, they also commonly deliberate over their options, thinking through potential outcomes and reflecting on their own preferences. In this Perspective, we bring to bear recent research into the higher-order capacities that underpin deliberation-particularly those that enable people to think about the future (prospection) and their own thinking (metacognition)-to shed light on intertemporal decision-making. We show how a greater appreciation for these mechanisms of deliberation promises to advance our understanding of intertemporal decision-making and unify a wide range of otherwise disparate choice phenomena.

Past behaviours and future intentions: An examination of perceptual deterrence and alcohol consumption upon a range of drink driving events

INTRODUCTION

The threat of application of legal sanctions remains the prominent approach to reduce the prevalence of drink driving in a vast array of motoring jurisdictions. However, ongoing questions remain regarding: (a) the extent that such mechanisms impact upon offending behaviours, (b) the deleterious effect alcohol consumption has on decisions to drink and drive and (c) how best to operationalise (and measure) the concept of drink driving to enhance the accurate measurement of the dependent variable.

METHOD

This paper reports on an examination of 773 Queensland motorists' (across nine local government areas) perceptions of both legal and non-legal drink driving sanctions (as well as alcohol consumption) in order to gauge the deterrent impact upon a range of measures of drink driving: the driver thinking they are over the limit, the driver knowing they are over the limit, attempts to evade random breath testing, and intentions to re-offend. The sample completed an online or paper version of the questionnaire.

RESULTS

The majority of participants reported "never" engaging in "possible" (74.5 %) or "acknowledged" (83.4 %) drink driving events, although a considerable proportion of the sample reported engaging in "possible" (25.5 %) or "acknowledged" (16.6 %) drink driving and attempting to evade RBT (18 %) events, as well as possible intentions to drink and drive in the future (22 %). Males were more likely to report such events. Perceptions of both legal sanctions (certainty, severity and swiftness) as well as non-legal sanctions (fear of social, internal or physical harm) were relatively high and consistent with previous research. Interestingly, non-legal sanctions were reported as stronger deterrents than legal sanctions. However, multivariate analysis revealed that legal deterrents had limited utility predicting offending behaviours, but rather, demographic characteristics (e.g., younger motorists, males) as well as risky drinking behaviour were better predictors. In regards to intentions to offend, a past conviction for drink driving was also a predictor of re-offending.

PRACTICAL APPLICATIONS

These results highlight the ongoing challenges of addressing the problem of drink driving and that some motorists: (a) have entrenched behaviour and/or (b) make the decision to drink and drive before they are under the influence of alcohol.

Subthreshold Fear Conditioning Produces a Rapidly Developing Neural Mechanism that Primes Subsequent Learning

Learning results in various forms of neuronal plasticity that provide a lasting representation of past events, and understanding the mechanisms supporting lasting memories has been a primary pursuit of the neurobiological study of memory. However, learning also alters the capacity for future learning, an observation that likely reflects its adaptive significance. In the laboratory, we can study this essential property of memory by assessing how prior experience alters the capacity for subsequent learning. Previous studies have indicated that while a single weak fear conditioning trial is insufficient to support long-term memory (LTM), it can facilitate future learning such that another trial delivered within a protracted time window results in a robust memory. Here, we sought to determine whether or not manipulating neural activity in the basolateral amygdala (BLA) using designer receptors exclusively activated by designer drugs (DREADDs) during or after the initial learning trial would affect the ability of the initial trial to facilitate subsequent learning. Our results show that inhibiting the BLA in rats prior to the first trial prevented the ability of that trial to facilitate learning when a second trial was presented the next day. Inhibition of the BLA immediately after the first trial using DREADDs was not effective, nor was pharmacological inhibition of protein kinase A (PKA) or the mitogen-activated protein kinase (MAPK). These findings indicate that the neural mechanisms that permit an initial subthreshold fear conditioning trial to alter later learning develop rapidly and do not appear to require a typical post-learning consolidation period.

Estimating Scale-Invariant Future in Continuous Time

Natural learners must compute an estimate of future outcomes that follow from a stimulus in continuous time. Widely used reinforcement learning algorithms discretize continuous time and estimate either transition functions from one step to the next (model-based algorithms) or a scalar value of exponentially discounted future reward using the Bellman equation (model-free algorithms). An important drawback of model-based algorithms is that computational cost grows linearly with the amount of time to be simulated. An important drawback of model-free algorithms is the need to select a timescale required for exponential discounting. We present a computational mechanism, developed based on work in psychology and neuroscience, for computing a scale-invariant timeline of future outcomes. This mechanism efficiently computes an estimate of inputs as a function of future time on a logarithmically compressed scale and can be used to generate a scale-invariant power-law-discounted estimate of expected future reward. The representation of future time retains information about what will happen when. The entire timeline can be constructed in a single parallel operation that generates concrete behavioral and neural predictions. This computational mechanism could be incorporated into future reinforcement learning algorithms.

"Surprise" and the Bayesian Brain: Implications for Psychotherapy Theory and Practice

The free energy principle (FEP) has gained widespread interest and growing acceptance as a new paradigm of brain function, but has had little impact on the theory and practice of psychotherapy. The aim of this paper is to redress this. Brains rely on Bayesian inference during which “bottom-up” sensations are matched with “top-down” predictions. Discrepancies result in “prediction error.” The brain abhors informational “surprise,” which is minimized by (1) action enhancing the statistical likelihood of sensory samples, (2) revising inferences in the light of experience, updating “priors” to reality-aligned “posteriors,” and (3) optimizing the complexity of our generative models of a capricious world. In all three, free energy is converted to bound energy. In psychopathology energy either remains unbound, as in trauma and inhibition of agency, or manifests restricted, anachronistic “top-down” narratives. Psychotherapy fosters client agency, linguistic and practical. Temporary uncoupling bottom-up from top-down automatism and fostering scrutinized simulations sets a number of salutary processes in train. Mentalising enriches Bayesian inference, enabling experience and feeling states to be “metabolized” and assimilated. “Free association” enhances more inclusive sensory sampling, while dream analysis foregrounds salient emotional themes as “attractors.” FEP parallels with psychoanalytic theory are outlined, including Freud’s unpublished project, Bion’s “contact barrier” concept, the Fonagy/Target model of sexuality, Laplanche’s therapist as “enigmatic signifier,” and the role of projective identification. The therapy stimulates patients to become aware of and revise the priors’ they bring to interpersonal experience. In the therapeutic “duet for one,” the energy binding skills and non-partisan stance of the analyst help sufferers face trauma without being overwhelmed by psychic entropy. Overall, the FEP provides a sound theoretical basis for psychotherapy practice, training, and research.

The Successor Representation: Its Computational Logic and Neural Substrates

Reinforcement learning is the process by which an agent learns to predict long-term future reward. We now understand a great deal about the brain's reinforcement learning algorithms, but we know considerably less about the representations of states and actions over which these algorithms operate. A useful starting point is asking what kinds of representations we would want the brain to have, given the constraints on its computational architecture. Following this logic leads to the idea of the successor representation, which encodes states of the environment in terms of their predictive relationships with other states. Recent behavioral and neural studies have provided evidence for the successor representation, and computational studies have explored ways to extend the original idea. This paper reviews progress on these fronts, organizing them within a broader framework for understanding how the brain negotiates tradeoffs between efficiency and flexibility for reinforcement learning.

Planning and navigation as active inference

This paper introduces an active inference formulation of planning and navigation. It illustrates how the exploitation-exploration dilemma is dissolved by acting to minimise uncertainty (i.e. expected surprise or free energy). We use simulations of a maze problem to illustrate how agents can solve quite complicated problems using context sensitive prior preferences to form subgoals. Our focus is on how epistemic behaviour-driven by novelty and the imperative to reduce uncertainty about the world-contextualises pragmatic or goal-directed behaviour. Using simulations, we illustrate the underlying process theory with synthetic behavioural and electrophysiological responses during exploration of a maze and subsequent navigation to a target location. An interesting phenomenon that emerged from the simulations was a putative distinction between 'place cells'-that fire when a subgoal is reached-and 'path cells'-that fire until a subgoal is reached.