Overlap in the functional neural systems involved in semantic and episodic memory retrieval

A method to fuse fMRI tasks through spatial correlations: applied to schizophrenia

Single task analysis methods of functional MRI brain data, though useful, are not able to evaluate the joint information between tasks. Data fusion of multiple tasks that probe different cognitive processes provides knowledge of the joint information and may be important in order to better understand complex disorders such as schizophrenia. In this article, we introduce a simple but effective technique to fuse two tasks by computing the histogram of correlations for all possible combinations of whole brain voxels. The approach was applied to data derived from healthy controls and patients with schizophrenia from four different tasks, auditory oddball (target), auditory oddball (novel), Sternberg working memory, and sensorimotor. It was found that in four out of six task combinations patients' intertask correlations were more positively correlated than controls', in one combination the controls showed more positive correlations and in another there was no significant difference. The robustness of this result was checked with several testing techniques. The four task combinations for which patients had more positive correlation occurred at different scanning sessions and the task combination that showed the opposite result occurred within the same scanning session. Brain regions that showed high intertask correlations were found for both groups and regions that correlated differently between the two groups were identified. The approach introduced finds interesting results and new differential features that cannot be achieved through traditional methods.

Contribution of prior semantic knowledge to new episodic learning in amnesia

We evaluated whether prior semantic knowledge would enhance episodic learning in amnesia. Subjects studied prices that are either congruent or incongruent with prior price knowledge for grocery and household items and then performed a forced-choice recognition test for the studied prices. Consistent with a previous report, healthy controls' performance was enhanced by price knowledge congruency; however, only a subset of amnesic patients experienced the same benefit. Whereas patients with relatively intact semantic systems, as measured by an anatomical measure (i.e., lesion involvement of anterior and lateral temporal lobes), experienced a significant congruency benefit, patients with compromised semantic systems did not experience a congruency benefit. Our findings suggest that when prior knowledge structures are intact, they can support acquisition of new episodic information by providing frameworks into which such information can be incorporated.

Linking functional and structural brain images with multivariate network analyses: a novel application of the partial least square method

In this article, we introduce a multimodal multivariate network analysis to characterize the linkage between the patterns of information from the same individual's complementary brain images, and illustrate its potential by showing its ability to distinguish older from younger adults with greater power than several previously established methods. Our proposed method uses measurements from every brain voxel in each person's complementary co-registered images and uses the partial least square (PLS) algorithm to form a combined latent variable that maximizes the covariance among all of the combined variables. It represents a new way to calculate the singular value decomposition from the high-dimensional covariance matrix in a computationally feasible way. Analyzing fluorodeoxyglucose positron emission tomography (PET) and volumetric magnetic resonance imaging (MRI) images, this method distinguished 14 older adults from 15 younger adults (p=4e-12) with no overlap between groups, no need to correct for multiple comparisons, and greater power than the univariate Statistical Parametric Mapping (SPM), multimodal SPM or multivariate PLS analysis of either imaging modality alone. This technique has the potential to link patterns of information among any number of complementary images from an individual, to use other kinds of complementary complex datasets besides brain images, and to characterize individual state- or trait-dependent brain patterns in a more powerful way.

Effects of semantic relatedness on recall of stimuli preceding emotional oddballs

Semantic and episodic memory networks function as highly interconnected systems, both relying on the hippocampal/medial temporal lobe complex (HC/MTL). Episodic memory encoding triggers the retrieval of semantic information, serving to incorporate contextual relationships between the newly acquired memory and existing semantic representations. While emotional material augments episodic memory encoding at the time of stimulus presentation, interactions between emotion and semantic memory that contribute to subsequent episodic recall are not well understood. Using a modified oddball task, we examined the modulatory effects of negative emotion on semantic interactions with episodic memory by measuring the free-recall of serially presented neutral or negative words varying in semantic relatedness. We found increased free-recall for words related to and preceding emotionally negative oddballs, suggesting that negative emotion can indirectly facilitate episodic free-recall by enhancing semantic contributions during encoding. Our findings demonstrate the ability of emotion and semantic memory to interact to mutually enhance free-recall.

Feature-based fusion of medical imaging data

The acquisition of multiple brain imaging types for a given study is a very common practice. There have been a number of approaches proposed for combining or fusing multitask or multimodal information. These can be roughly divided into those that attempt to study convergence of multimodal imaging, for example, how function and structure are related in the same region of the brain, and those that attempt to study the complementary nature of modalities, for example, utilizing temporal EEG information and spatial functional magnetic resonance imaging information. Within each of these categories, one can attempt data integration (the use of one imaging modality to improve the results of another) or true data fusion (in which multiple modalities are utilized to inform one another). We review both approaches and present a recent computational approach that first preprocesses the data to compute features of interest. The features are then analyzed in a multivariate manner using independent component analysis. We describe the approach in detail and provide examples of how it has been used for different fusion tasks. We also propose a method for selecting which combination of modalities provides the greatest value in discriminating groups. Finally, we summarize and describe future research topics.

Hippocampal activation during episodic and semantic memory retrieval: comparing category production and category cued recall

Whether or not the hippocampus participates in semantic memory retrieval has been the focus of much debate in the literature. However, few neuroimaging studies have directly compared hippocampal activation during semantic and episodic retrieval tasks that are well matched in all respects other than the source of the retrieved information. In Experiment 1, we compared hippocampal fMRI activation during a classic semantic memory task, category production, and an episodic version of the same task, category cued recall. Left hippocampal activation was observed in both episodic and semantic conditions, although other regions of the brain clearly distinguished the two tasks. Interestingly, participants reported using retrieval strategies during the semantic retrieval task that relied on autobiographical and spatial information; for example, visualizing themselves in their kitchen while producing items for the category kitchen utensils. In Experiment 2, we considered whether the use of these spatial and autobiographical retrieval strategies could have accounted for the hippocampal activation observed in Experiment 1. Categories were presented that elicited one of three retrieval strategy types, autobiographical and spatial, autobiographical and nonspatial, and neither autobiographical nor spatial. Once again, similar hippocampal activation was observed for all three category types, regardless of the inclusion of spatial or autobiographical content. We conclude that the distinction between semantic and episodic memory is more complex than classic memory models suggest.

Common and Unique Neural Activations in Autobiographical, Episodic, and Semantic Retrieval

This study sought to explore the neural correlates that underlie autobiographical, episodic, and semantic memory. Autobiographical memory was defined as the conscious recollection of personally relevant events, episodic memory as the recall of stimuli presented in the laboratory, and semantic memory as the retrieval of factual information and general knowledge about the world. Our objective was to delineate common neural activations, reflecting a functional overlap, and unique neural activations, reflecting functional dissociation of these memory processes. We conducted an event-related functional magnetic resonance imaging study in which we utilized the same pictorial stimuli but manipulated retrieval demands to extract autobiographical, episodic, or semantic memories. The results show a functional overlap of the three types of memory retrieval in the inferior frontal gyrus, the middle frontal gyrus, the caudate nucleus, the thalamus, and the lingual gyrus. All memory conditions yielded activation of the left medial-temporal lobe; however, we found a functional dissociation within this region. The anterior and superior areas were active in episodic and semantic retrieval, whereas more posterior and inferior areas were active in autobiographical retrieval. Unique activations for each memory type were also delineated, including medial frontal increases for autobiographical, right middle frontal increases for episodic, and right inferior temporal increases for semantic retrieval. These findings suggest a common neural network underlying all declarative memory retrieval, as well as unique neural contributions reflecting the specific properties of retrieved memories.

Distinguishing the neural correlates of episodic memory encoding and semantic memory retrieval

Episodic memory and semantic memory interact very closely. In particular, episodic memory encoding (EE) tends to elicit semantic memory retrieval (SR), and vice versa. Thus, similar activations for EE and SR in functional neuroimaging studies may reflect shared memory processes, or they may reflect the fact that EE and SR are usually confounded. To address this issue, we used a factorial functional magnetic resonance imaging approach to disentangle the neural correlates of EE and SR. Within the left temporal lobe, the hippocampus was associated with successful EE, whereas a posterior lateral region was associated with successful SR. Within the left inferior prefrontal cortex, a posterior region was involved in SR, a mid region was involved in both SR and EE, and an anterior region was involved in EE, but only when SR was also high. Thus, the neural correlates of EE and SR are dissociable but interact in specific brain regions.

Inhibition of return impairs episodic memory access

The term inhibition of return (IOR) refers to a bias against returning attention to a location or object that has recently been attended. The effect has been shown to occur in various perceptual tasks including stimulus detection, localization, and discrimination, but also to affect higher cognitive processes like lexical access. The present experiments examined whether inhibition of return would impair the high-level processing that is required in accessing item representations in episodic memory. The results show that reaction times for recognition memory decisions are increased under IOR. Furthermore, IOR affects the accuracy of recognition memory, and this effect interacts with the ease of memory access, manipulated, for example, by encoding depth in the learning phase. These results suggest that IOR impairs attentional processing up to the highest cognitive levels, including the access of prior item encounters in episodic memory.

Prefrontal and hippocampal contributions to the generation and binding of semantic associations during successful encoding

The ability to form and bind associations between items is an important aspect of successful memory formation. We hypothesize that, during encoding, the left inferior frontal gyrus (IFG) supports generation of associations between items and the hippocampus then binds these associations. This study examined the parametric responses of these regions to varying amounts of generative and relational processing during successful encoding (i.e., for subsequently recognized items). Encoding involved presentation of word triads varying in the number of semantic associations among them (none, one or all); participants judged how many associations were present in each triad. Thus, triads with fewer associations had higher generative load while triads with more associations had higher relational load. Participants later completed a forced-choice recognition test for encoding triads. Successful encoding relative to a control task resulted in activation of bilateral IFG and left hippocampus, and the hippocampus also exhibited a significant subsequent memory effect (hits>misses). Linear parametric analyses revealed that generative load modulated activity in bilateral IFG while relational load correlated with activity in left hippocampus. Although univariate analyses distinguished IFG and hippocampal contributions to the generative and relational stages of encoding, respectively, effective connectivity between these regions did not differ according to condition. Furthermore, this analysis revealed that the left IFG played a pivotal role in coordinating associative encoding processes. Our findings illustrate that modulation of components in a memory network can be independent of patterns of mutual connectivity among those components in mediating successful encoding.

A method for multitask fMRI data fusion applied to schizophrenia

It is becoming common to collect data from multiple functional magnetic resonance imaging (fMRI) paradigms on a single individual. The data from these experiments are typically analyzed separately and sometimes directly subtracted from one another on a voxel-by-voxel basis. These comparative approaches, although useful, do not directly attempt to examine potential commonalities between tasks and between voxels. To remedy this we propose a method to extract maximally spatially independent maps for each task that are "coupled" together by a shared loading parameter. We first compute an activation map for each task and each individual as "features," which are then used to perform joint independent component analysis (jICA) on the group data. We demonstrate our approach on a data set derived from healthy controls and schizophrenia patients, each of which carried out an auditory oddball task and a Sternberg working memory task. Our analysis approach revealed two interesting findings in the data that were missed with traditional analyses. First, consistent with our hypotheses, schizophrenia patients demonstrate "decreased" connectivity in a joint network including portions of regions implicated in two prevalent models of schizophrenia. A second finding is that for the voxels identified by the jICA analysis, the correlation between the two tasks was significantly higher in patients than in controls. This finding suggests that schizophrenia patients activate "more similarly" for both tasks than do controls. A possible synthesis of both findings is that patients are activating less, but also activating with a less-unique set of regions for these very different tasks. Both of the findings described support the claim that examination of joint activation across multiple tasks can enable new questions to be posed about fMRI data. Our approach can also be applied to data using more than two tasks. It thus provides a way to integrate and probe brain networks using a variety of tasks and may increase our understanding of coordinated brain networks and the impact of pathology upon them.