The Generalized Quantum Episodic Memory Model

Temporally extended successor feature neural episodic control

One of the long-term goals of reinforcement learning is to build intelligent agents capable of rapidly learning and flexibly transferring skills, similar to humans and animals. In this paper, we introduce an episodic control framework based on the temporal expansion of subsequent features to achieve these goals, which we refer to as Temporally Extended Successor Feature Neural Episodic Control (TESFNEC). This method has shown impressive results in significantly improving sample efficiency and elegantly reusing previously learned strategies. Crucially, this model enhances agent training by incorporating episodic memory, significantly reducing the number of iterations required to learn the optimal policy. Furthermore, we adopt the temporal expansion of successor features a technique to capture the expected state transition dynamics of actions. This form of temporal abstraction does not entail learning a top-down hierarchy of task structures but focuses on the bottom-up combination of actions and action repetitions. Thus, our approach directly considers the temporal scope of sequences of temporally extended actions without requiring predefined or domain-specific options. Experimental results in the two-dimensional object collection environment demonstrate that the method proposed in this paper optimizes learning policies faster than baseline reinforcement learning approaches, leading to higher average returns.

Temporal Bell inequalities in cognition

There is widespread evidence that human memory is constructive, so that recollective processes may alter the information retrieved or impact on subsequent recollections. We examine a framework for narrowing down the nature of such processes, from physics. In Physics, the Temporal Bell (TB) inequality offers a general test of the sensitivity of the context of previous measurements in sequential measurement scenarios, as predicted by quantum theory. We present an empirical memory paradigm that allows a test of the TB inequality, using a novel kind of "change judgment," whereby participants are asked to decide whether there has been a change in a question across different time points of a scenario. Across two experiments, we were able to observe evidence for the violation of a TB inequality in one case, offering evidence for quantum-like processes in memory. The present results complement other recent work purporting the relevance of quantum-like representations in memory and raise questions regarding the adaptive value of such representations.

Quantum Cognition

Uncertainty is an intrinsic part of life; most events, affairs, and questions are uncertain. A key problem in behavioral sciences is how the mind copes with uncertain information. Quantum probability theory offers a set of principles for inference, which align well with intuition about psychological processes in certain cases: cases when it appears that inference is contextual, the mental state changes as a result of previous judgments, or there is interference between different possibilities. We motivate the use of quantum theory in cognition and its key characteristics. For each of these characteristics, we review relevant quantum cognitive models and empirical support. The scope of quantum cognitive models encompasses fallacies in decision-making (such as the conjunction fallacy or the disjunction effect), question order effects, conceptual combination, evidence accumulation, perception, over-/underdistribution effects in memory, and more. Quantum models often formalize psychological ideas previously expressed in heuristic terms, allow unified explanations of previously disparate findings, and have led to several surprising, novel predictions. We also cast a critical eye on quantum models and consider some of their shortcomings and issues regarding their further development. Expected final online publication date for the Annual Review of Psychology, Volume 73 is January 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Deep memory distortions

Deep distortions are a new family of memory biases that comprise one of the two basic varieties of false memory. The first and older variety, surface distortions, are specific item or source memories that are erroneous because the events did not happen. The new variety, deep distortions, are emergent properties of multiple specific memories. They are relations among such memories that are false because they violate objective logical rules that real-world events must obey. I discuss four deep distortions for which substantial data have accumulated: overdistribution, super-overdistribution, non-additivity, and impossible conjunctions. These phenomena violate four axioms of classical probability (numerical bound, universal event, additivity, and countable additivity) and two rules that follow from them (empty set and monotonicity). Their psychological significance lies in four facts about them: (a) They demonstrate that although events in the real world are compensatory, our memories of them are not; (b) they establish that we persistently over remember experience; (c) they reveal that surface distortions are by-products of deep distortions; and (d) they pose the theoretical conundrum of how the structure of memory could so thoroughly misrepresent the objective structure of the events we are remembering.

Deep distortion

Conventional false memories recount events that either did not happen (item errors) or that happened in a different context (source errors). Fuzzy-trace theory predicts deeper anomalies that lie behind conventional false memories. These deep distortions are structural irregularities in the ways that specific recountings are related to each other or to some objective standard (e.g., the 0 and 1 limits of probability). I discuss five deep distortions for which substantial data have accumulated: overdistribution, non-additivity, conjunction illusions, non-compensation, and super-overdistribution. Together, these phenomena violate the disjunction and additivity axioms of probability, as well as the law of the excluded middle. The theoretical problem they pose is to explain how valid representations of our experience produce memory regularities that violate our experience in the most fundamental ways.