Neural correlates of state- and strength-based perception

The role of recollection, familiarity, and the hippocampus in episodic and working memory

The hippocampus plays an essential role in long-term episodic memory by supporting the recollection of contextual details, whereas surrounding regions such as the perirhinal cortex support familiarity-based recognition discriminations. Working memory - the ability to maintain information over very brief periods of time - is traditionally thought to rely heavily on frontoparietal attention networks, but recent work has shown that it can also rely on the hippocampus. However, the conditions in which the hippocampus becomes involved in working memory tasks are unclear and whether it contributes to recollection or familiarity-based responses in working memory is only beginning to be explored. In the current paper, we first review and contrast the existing amnesia literature examining recollection and familiarity in episodic and working memory. The results indicate that recollection and familiarity contribute to both episodic and working memory. However, in contrast to episodic memory, in working memory the hippocampus is particularly critical for familiarity-based rather than recollection-based discrimination. Moreover, the results indicate that the role of the hippocampus in working memory can be obscured due to 'criterion-induced process-masking' because it primarily supports intermediate-confidence recognition decisions. We then report results from a new working memory study examining the ability of amnesics to detect global and local changes in novel complex objects (i.e., fribbles), which indicates that the hippocampus plays an especially critical role in working memory when the task requires the detection of global rather than discrete changes. We conclude by considering the results in light of neurocomputational models and proposing a general framework for understanding the relationship between episodic and working memory.

Eye movements dissociate between perceiving, sensing, and unconscious change detection in scenes

Detecting visual changes can be based on perceiving, whereby one can identify a specific detail that has changed, on sensing, whereby one knows that there is a change but is unable to identify what changed, or on unconscious change detection, whereby one is unaware of any change even though the change influences one's behavior. Prior work has indicated that the processes underlying these different types of change detection are functionally and neurally distinct, but the attentional mechanisms that are related to these different types of change detection remain largely unknown. In the current experiment, we examined eye movements during a change detection task in globally manipulated scenes, and participants indicated their change detection confidence on a scale that allowed us to isolate perceiving, sensing, and unconscious change detection. For perceiving-based change detection, but not sensing-based or unconscious change detection, participants were more likely to preferentially revisit highly changed scene regions across the first and second presentation of the scene (i.e., resampling). This increase in resampling started within 250 ms of the test scene onset, suggesting that the effect began within the first two fixations. In addition, changed scenes were related to more clustered (i.e., less dispersed) eye movements than unchanged scenes, particularly when the subjects were highly confident that no change had occurred - providing evidence for change detection outside of conscious awareness. The results indicate that perceiving, sensing, and unconscious change detection responses are related to partially distinct patterns of eye movements.

Early Visual Processing and Perception Processes in Object Discrimination Learning

A brief image presentation is sufficient to discriminate and individuate objects of expertise. Although perceptual expertise is acquired through extensive practice that increases the resolution of representations and reduces the latency of image decoding and coarse and fine information extraction, it is not known how the stages of visual processing impact object discrimination learning (ODL). Here, we compared object discrimination with brief (100 ms) and long (1,000 ms) perceptual encoding times to test if the early and late visual processes are required for ODL. Moreover, we evaluated whether encoding time and discrimination practice shape perception and recognition memory processes during ODL. During practice of a sequential matching task with initially unfamiliar complex stimuli, we find greater discrimination with greater encoding times regardless of the extent of practice, suggesting that the fine information extraction during late visual processing is necessary for discrimination. Interestingly, the overall discrimination learning was similar for brief and long stimuli, suggesting that early stages of visual processing are sufficient for ODL. In addition, discrimination practice enhances perceive and know for brief and long stimuli and both processes are associated with performance, suggesting that early stage information extraction is sufficient for modulating the perceptual processes, likely reflecting an increase in the resolution of the representations and an early availability of information. Conversely, practice elicited an increase of familiarity which was not associated with discrimination sensitivity, revealing the acquisition of a general recognition memory. Finally, the recall is likely enhanced by practice and is associated with discrimination sensitivity for long encoding times, suggesting the engagement of recognition memory in a practice independent manner. These findings contribute to unveiling the function of early stages of visual processing in ODL, and provide evidence on the modulation of the perception and recognition memory processes during discrimination practice and its relationship with ODL and perceptual expertise acquisition.

Visual working memory impairments for single items following medial temporal lobe damage

A growing body of research indicates that the medial temporal lobe (MTL) is essential not only for long-term episodic memory but also for visual working memory (VWM). In particular, recent work has shown that the MTL is especially important for VWM when complex, high-resolution binding is required. However, all of these studies tested VWM for multiple items which invites the possibility that working memory capacity was exceeded and patient impairments instead reflected deficits in long-term memory. Thus, the precise conditions under which the MTL is critical for VWM and the type of working memory processes that are affected by MTL damage are not yet clear. To address these issues, we examined the effects of MTL damage on VWM for a single item (i.e., a square that contained color, location, and orientation information) using confidence-based receiver operating characteristic methods to assess VWM discriminability and to separate perceiving- and sensing-based memory judgments. This approach was motivated by dual-process theories of cognition that posit distinct subprocesses underlie performance across perception, working memory, and long-term memory. The results indicated that MTL patients were significantly impaired in VWM for a single item. Interestingly, the patients were not impaired at making accurate high-confidence judgments that a change had occurred (i.e., perceiving), rather they were impaired at making low-confidence judgments that they sensed whether or not there had been a change in the absence of identifying the exact change. These results demonstrate that the MTL is critical in supporting working memory even for a single item, and that it contributes selectively to sensing-based discriminations.

The effects of face inversion on perceiving- and sensing-based change detection

Face perception is more difficult when faces are inverted compared to when they are upright. However, it is not known whether face inversion disrupts the ability to make perceiving-based discriminations (i.e., the ability to identify a specific feature change), or sensing-based discriminations (i.e., the ability to detect there was a change without the ability to identify what changed). In the current study, we used confidence-based receiver operating characteristics (ROCs) in a change detection test to examine the effect of face inversion on perceiving and sensing. In Experiment 1, face inversion led to a reduction in the probability of perceiving but did not impact sensing-based discriminations. In Experiment 2, we replicated these results, and verified that the findings based on ROC estimates paralleled participants' phenomenological experiences of perceiving and sensing. Furthermore, the perceiving-based face inversion effect was found to reflect a reduction in the ability to accurately report specific feature changes. These findings indicate that face inversion does not reduce the ability to sense there was a change in the absence of identification, but rather it reduces the ability to consciously identify specific characteristics of faces in service of perceiving-based discriminations. In addition, they suggest that sensing responds to global differences across the visual image, rather than to changes in holistic processing of the visual input. These results further our understanding of the face inversion effect and clarify the nature of the processes underlying visual perception. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Change detection on a hunch: pre-attentive vision allows "sensing" of unique feature changes

Studies on change detection and change blindness have investigated the nature of visual representations by testing the conditions under which observers are able to detect when an object in a complex scene changes from one moment to the next. Several authors have proposed that change detection can occur without identification of the changing object, but the perceptual processes underlying this phenomenon are currently unknown. We hypothesized that change detection without localization or identification occurs when the change happens outside the focus of attention. Such changes would usually go entirely unnoticed, unless the change brings about a modification of one of the feature maps representing the scene. Thus, the appearance or disappearance of a unique feature might be registered even in the absence of focused attention and without feature binding, allowing for change detection, but not localization or identification. We tested this hypothesis in three experiments, in which changes either involved colors that were already present elsewhere in the display or entirely unique colors. Observers detected whether any change had occurred and then localized or identified the change. Change detection without localization occurred almost exclusively when changes involved a unique color. Moreover, change detection without localization for unique feature changes was independent of the number of objects in the display and independent of change identification. These findings suggest that pre-attentive registration of a change on a feature map can give rise to a conscious experience even when feature binding has failed: that something has changed without knowing what or where.

Cortical and subcortical contributions to state- and strength-based perceptual judgments

Perceptual judgments can be made on the basis of different kinds of information: state-based access to specific details that differentiate two similar images, or strength-based assessments of relational match/mismatch. We explored state- and strength-based perception in eleven right-hemisphere stroke patients, and examined lesion overlap images to gain insight into the neural underpinnings of these different kinds of perceptual judgments. Patients and healthy controls were presented with pairs of scenes that were either identical or differed in that one scene was slightly expanded or contracted relative to the other. Same/different confidence judgments were used to plot receiver-operating characteristics and estimate the contributions of state- and strength-based perception. The patient group showed a significant and selective impairment of strength-based, but not state-based, perception. This finding was not an artifact of reduced levels of overall performance, because matching perceptual discriminability levels between controls and patients revealed a double dissociation, with higher state-based, and lower strength-based, perception in patients vs.

CONTROLS

We then conducted exploratory follow-up analyses on the patient group, based on the observation of substantial individual differences in state-based perception - differences that were masked in analyses based on the group mean. Patients who were relatively spared in state-based perception (but impaired in strength-based perception) had damage that was primarily in temporo-parietal cortical regions. Patients who were relatively impaired in both state- and strength-based perception had overlapping damage in the thalamus, putamen, and adjacent white matter. These patient groups were not different in any other measure, e.g., presence of spatial neglect symptoms, age, education, lesion volume, or time since stroke. These findings shed light on the different roles of right hemisphere regions in high-level perception, suggesting that the thalamus and basal ganglia play a critical role in state- and strength-based perception, whereas temporo-parietal cortical regions are important for intact strength-based perception.

Neurocomputational account of memory and perception: Thresholded and graded signals in the hippocampus

Recent evidence suggests that the hippocampus, a region critical for long-term memory, also supports certain forms of high-level visual perception. A seemingly paradoxical finding is that, unlike the thresholded hippocampal signals associated with memory, the hippocampus produces graded, strength-based signals in perception. This article tests a neurocomputational model of the hippocampus, based on the complementary learning systems framework, to determine if the same model can account for both memory and perception, and whether it produces the appropriate thresholded and strength-based signals in these two types of tasks. The simulations showed that the hippocampus, and most prominently the CA1 subfield, produced graded signals when required to discriminate between highly similar stimuli in a perception task, but generated thresholded patterns of activity in recognition memory. A threshold was observed in recognition memory because pattern completion occurred for only some trials and completely failed to occur for others; conversely, in perception, pattern completion always occurred because of the high degree of item similarity. These results offer a neurocomputational account of the distinct hippocampal signals associated with perception and memory, and are broadly consistent with proposals that CA1 functions as a comparator of expected versus perceived events. We conclude that the hippocampal computations required for high-level perceptual discrimination are congruous with current neurocomputational models that account for recognition memory, and fit neatly into a broader description of the role of the hippocampus for the processing of complex relational information.