Object activation in semantic memory from visual multimodal feature input

A modified neural circuit framework for semantic memory retrieval with implications for circuit modulation to treat verbal retrieval deficits

Word finding difficulty is a frequent complaint in older age and disease states, but treatment options are lacking for such verbal retrieval deficits. Better understanding of the neurophysiological and neuroanatomical basis of verbal retrieval function may inform effective interventions. In this article, we review the current evidence of a neural retrieval circuit central to verbal production, including words and semantic memory, that involves the pre-supplementary motor area (pre-SMA), striatum (particularly caudate nucleus), and thalamus. We aim to offer a modified neural circuit framework expanded upon a memory retrieval model proposed in 2013 by Hart et al., as evidence from electrophysiological, functional brain imaging, and noninvasive electrical brain stimulation studies have provided additional pieces of information that converge on a shared neural circuit for retrieval of memory and words. We propose that both the left inferior frontal gyrus and fronto-polar regions should be included in the expanded circuit. All these regions have their respective functional roles during verbal retrieval, such as selection and inhibition during search, initiation and termination of search, maintenance of co-activation across cortical regions, as well as final activation of the retrieved information. We will also highlight the structural connectivity from and to the pre-SMA (e.g., frontal aslant tract and fronto-striatal tract) that facilitates communication between the regions within this circuit. Finally, we will discuss how this circuit and its correlated activity may be affected by disease states and how this circuit may serve as a novel target engagement for neuromodulatory treatment of verbal retrieval deficits.

Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models

Introduction

Associating multimodal information is essential for human cognitive abilities including mathematical skills. Multimodal learning has also attracted attention in the field of machine learning, and it has been suggested that the acquisition of better latent representation plays an important role in enhancing task performance. This study aimed to explore the impact of multimodal learning on representation, and to understand the relationship between multimodal representation and the development of mathematical skills.

Methods

We employed a multimodal deep neural network as the computational model for multimodal associations in the brain. We compared the representations of numerical information, that is, handwritten digits and images containing a variable number of geometric figures learned through single- and multimodal methods. Next, we evaluated whether these representations were beneficial for downstream arithmetic tasks.

Results

Multimodal training produced better latent representation in terms of clustering quality, which is consistent with previous findings on multimodal learning in deep neural networks. Moreover, the representations learned using multimodal information exhibited superior performance in arithmetic tasks.

Discussion

Our novel findings experimentally demonstrate that changes in acquired latent representations through multimodal association learning are directly related to cognitive functions, including mathematical skills. This supports the possibility that multimodal learning using deep neural network models may offer novel insights into higher cognitive functions.

Thalamic Aphasia: a Review

PURPOSE OF REVIEW

Thalamic aphasia is a rare language disorder resulting from lesions to the thalamus. While most patients exhibit mild symptoms with a predominance of lexical-semantic difficulties, variations in phenotype have been described. Overall, the exact mechanisms of thalamic aphasia await empirical research. The article reviews recent findings regarding phenotypes and possible underlying mechanisms of thalamic aphasia.

RECENT FINDINGS

Variations in phenotype of thalamic aphasia may be related to different lesion locations. Overall, the thalamus' role in language is thought to be due to its involvement in cortico-thalamic language networks with lesioning of certain nuclei resulting in the diachisis of otherwise interconnected areas. Its possible monitoring function in such a network might be due to its different cellular firing modes. However, no specific evidence has been collected to date. While recent findings show a more distinct understanding of thalamic aphasia phenotypes and possible underlying mechanisms, further research is needed. Additionally, as standard language testing might oftentimes not pick up on its subtle symptoms, thalamic aphasia might be underdiagnosed.

The Role of the Thalamus in Declarative and Procedural Linguistic Memory Processes

Typically, thalamic aphasias appear to be primarily lexical-semantic disorders representing difficulty using stored declarative memories for semantic information to access lexical word forms. Yet, there also is reason to believe that the thalamus might play a role in linguistic procedural memory. For more than two decades, we have known that basal ganglia dysfunction is associated with difficulties in procedural learning, and specific thalamic nuclei are the final waypoint back to the cortex in cortico-basal ganglia-cortical loops. Recent analyses of the role of the thalamus in lexical-semantic processes and of the role of the basal ganglia in linguistic processes suggest that thalamic participation is not simply a matter of declarative vs. procedural memory, but a matter of how the thalamus participates in lexical-semantic processes and in linguistic procedural memory, as well as the interaction of these processes. One role for the thalamus in accessing lexical forms for semantic concepts relates to the stabilization of a very complex semantic-lexical interface with thousands of representations on both sides of the interface. Further, the possibility is discussed that the thalamus, through its participation in basal ganglia loops, participates in two linguistic procedural memory processes: syntactic/grammatical procedures and procedures for finding words to represent semantic concepts, with the latter interacting intricately with declarative memories. These concepts are discussed in detail along with complexities that can be addressed by future research.

The visual and semantic features that predict object memory: Concept property norms for 1,000 object images

Humans have a remarkable fidelity for visual long-term memory, and yet the composition of these memories is a longstanding debate in cognitive psychology. While much of the work on long-term memory has focused on processes associated with successful encoding and retrieval, more recent work on visual object recognition has developed a focus on the memorability of specific visual stimuli. Such work is engendering a view of object representation as a hierarchical movement from low-level visual representations to higher level categorical organization of conceptual representations. However, studies on object recognition often fail to account for how these high- and low-level features interact to promote distinct forms of memory. Here, we use both visual and semantic factors to investigate their relative contributions to two different forms of memory of everyday objects. We first collected normative visual and semantic feature information on 1,000 object images. We then conducted a memory study where we presented these same images during encoding (picture target) on Day 1, and then either a Lexical (lexical cue) or Visual (picture cue) memory test on Day 2. Our findings indicate that: (1) higher level visual factors (via DNNs) and semantic factors (via feature-based statistics) make independent contributions to object memory, (2) semantic information contributes to both true and false memory performance, and (3) factors that predict object memory depend on the type of memory being tested. These findings help to provide a more complete picture of what factors influence object memorability. These data are available online upon publication as a public resource.

The Role of Cortico-Thalamo-Cortical Circuits in Language: Recurrent Circuits Revisited

Based on a review of recent literature, a recurrent circuit model describes how cortico-thalamo-cortical and cortico-cortical circuitry supports word retrieval, auditory-verbal comprehension, and other language functions. Supporting data include cellular and layer-specific cortico-thalamic, thalamo-cortical, and cortico-cortical neuroanatomy and electrophysiology. The model posits that during word retrieval, higher order cortico-thalamo-cortical relays maintain stable representations of semantic information in feedforward processes at the semantic-lexical interface. These stable semantic representations are compared to emerging lexical solutions to represent the semantic construct to determine how well constructs are associated with each other. The resultant error signal allows cortico-cortical sculpting of activity between the semantic and lexical mechanisms until there is a good match between these two levels, at which time the lexical solution will be passed along to the cortical processor necessary for the next stage of word retrieval. Evidence is cited that high gamma activity is the neural signature for processing in the cortico-thalamo-cortical and cortico-cortical circuitry. Methods for testing hypotheses generated from this recurrent circuit model are discussed. Mathematical modeling may be a useful tool in exploring underlying properties of these circuits.

Where are aphasia theory and management "headed"?

The sequelae of post-stroke aphasia are considerable, necessitating an understanding of the functional neuroanatomy of language, cognitive processes underlying various language tasks, and the mechanisms of recovery after stroke. This knowledge is vital in providing optimal care of individuals with aphasia and counseling to their families and caregivers. The standard of care in the rehabilitation of aphasia dictates that treatment be evidence-based and person-centered. Promising techniques, such as cortical stimulation as an adjunct to behavioral therapy, are just beginning to be explored. These topics are discussed in this review.

Electrophysiological correlates of semantic memory retrieval in Gulf War Syndrome 2 patients

Gulf War veterans meeting criteria for Haley Syndrome 2 of Gulf War illness endorse a particular constellation of symptoms that include difficulty with processing information, word-finding, and confusion. To explore the neural basis of their word-finding difficulty, we assessed event-related potentials (ERPs) associated with semantic memory retrieval in 22 veterans classified as Syndrome 2 and 28 veterans who served as controls. We recorded EEGs while subjects judged whether pairs of words that represented object features combined to elicit a retrieval of an object memory or no retrieval. Syndrome 2 subjects' responses were significantly slower, and those participants were less accurate than controls on the retrieval trials, but they performed similarly on the nonretrieval trials. Analysis of the ERPs revealed a difference between retrievals and nonretrievals that has previously been detected around 750ms at the left temporal region was present in both the Syndrome 2 patients and controls. However, the Syndrome 2 patients also showed an ERP difference between retrievals and nonretrievals at the midline parietal region that had a scalp voltage polarity opposite from that recorded at the left temporal area. We hypothesize that the similarities between task performance and ERP patterns in Syndrome 2 veterans and in patients with amnestic mild cognitive impairment reflect disordered thalamic cholinergic neural activity, possibly in the dorsomedial nucleus.

Thalamic deep brain stimulation decelerates automatic lexical activation

BACKGROUND

Deep Brain Stimulation (DBS) of the thalamic ventral intermediate nucleus (VIM) is a therapeutic option for patients with essential tremor. Despite a generally low risk of side effects, declines in verbal fluency (VF) have previously been reported.

OBJECTIVES

We aimed to specify effects of VIM-DBS on major cognitive operations needed for VF task performance, represented by clusters and switches. Clusters are word production spurts, thought to arise from automatic activation of associated information pertaining to a given lexical field. Switches are slow word-to-word transitions, presumed to indicate controlled operations for stepping from one lexical field to another.

PATIENTS & METHODS

Thirteen essential tremor patients with VIM-DBS performed verbal fluency tasks in their VIM-DBS ON and OFF conditions. Clusters and switches were formally defined by mathematical criteria. All results were compared to those of fifteen healthy control subjects, and significant OFF-ON-change scores were correlated to stimulation parameters.

RESULTS

Patients produced fewer words than healthy controls. DBS ON compared to DBS OFF aggravated this deficit by prolonging the intervals between words within clusters, whereas switches remained unaffected. This stimulation effect correlated with more anterior electrode positions.

CONCLUSION

VIM-DBS seems to influence word output dynamics during verbal fluency tasks on the level of word clustering. This suggests a perturbation of automatic lexical co-activation by thalamic stimulation, particularly if delivered relatively anteriorly. The findings are discussed in the context of the hypothesized role of the thalamus in lexical processing.

Sentence processing in anterior superior temporal cortex shows a social-emotional bias

The anterior region of the left superior temporal gyrus/superior temporal sulcus (aSTG/STS) has been implicated in two very different cognitive functions: sentence processing and social-emotional processing. However, the vast majority of the sentence stimuli in previous reports have been of a social or social-emotional nature suggesting that sentence processing may be confounded with semantic content. To evaluate this possibility we had subjects read word lists that differed in phrase/constituent size (single words, 3-word phrases, 6-word sentences) and semantic content (social-emotional, social, and inanimate objects) while scanned in a 7T environment. This allowed us to investigate if the aSTG/STS responded to increasing constituent structure (with increased activity as a function of constituent size) with or without regard to a specific domain of concepts, i.e., social and/or social-emotional content. Activity in the left aSTG/STS was found to increase with constituent size. This region was also modulated by content, however, such that social-emotional concepts were preferred over social and object stimuli. Reading also induced content type effects in domain-specific semantic regions. Those preferring social-emotional content included aSTG/STS, inferior frontal gyrus, posterior STS, lateral fusiform, ventromedial prefrontal cortex, and amygdala, regions included in the "social brain", while those preferring object content included parahippocampal gyrus, retrosplenial cortex, and caudate, regions involved in object processing. These results suggest that semantic content affects higher-level linguistic processing and should be taken into account in future studies.

Effects of thalamic deep brain stimulation on spontaneous language production

The thalamus is thought to contribute to language-related processing, but specifications of this notion remain vague. An assessment of potential effects of thalamic deep brain stimulation (DBS) on spontaneous language may help to delineate respective functions. For this purpose, we analyzed spontaneous language samples from thirteen (six female / seven male) patients with essential tremor treated with DBS of the thalamic ventral intermediate nucleus (VIM) in their respective ON vs. OFF conditions. Samples were obtained from semi-structured interviews and examined on multidimensional linguistic levels. In the VIM-DBS ON condition, participants used a significantly higher proportion of paratactic as opposed to hypotactic sentence structures. This increase correlated negatively with the change in the more global cognitive score, which in itself did not change significantly. In conclusion, VIM-DBS appears to induce the use of a simplified syntactic structure. The findings are discussed in relation to concepts of thalamic roles in language-related cognitive behavior.

Update in Aphasia Research

The sequelae of post-stroke aphasia are considerable, with implications at the societal and personal levels. An understanding of the mechanisms of recovery of cognitive and language processes after stroke and the factors associated with increased risk of post-stroke language and cognitive deficits is vital in providing optimal care of individuals with aphasia and in counseling to their families and caregivers. Advances in neuroimaging facilitate the identification of dysfunctional or damaged brain tissue responsible for these cognitive/language deficits and contribute insights regarding the functional neuroanatomy of language. Evidence-based person-centered behavioral therapy remains the mainstay for rehabilitation of aphasia, although emerging evidence shows that neuromodulation is a promising adjunct to traditional therapy. These topics are discussed in this review, illustrating with recent studies from the Stroke Cognitive Outcomes and REcovery (SCORE) lab.

A cortical-subcortical syntax pathway linking Broca's area and the striatum

Combinatorial syntax has been shown to be underpinned by cortical key regions such as Broca's area and temporal cortices, and by subcortical structures such as the striatum. The cortical regions are connected via several cortico-to-cortical tracts impacting syntactic processing (e.g., the arcuate) but it remains unclear whether and how the striatum can be integrated into this cortex-centered syntax network. Here, we used a systematic stepwise approach to investigate the existence and syntactic function of an additional deep Broca-striatum pathway. We first asked 15 healthy controls and 12 patients with frontal/striatal lesions to perform three syntax tests. The results obtained were subjected to voxel-based lesion-symptom mapping (VLSM) to provide an anatomo-functional approximation of the pathway. The significant VLSM clusters were then overlapped with the probability maps of four cortico-cortical language tracts generated for 12 healthy participants (arcuate, extreme capsule fiber system, uncinate, aslant), including a probabilistic Broca-striatum tract. Finally, we carried out quantitative analyses of the relationship between the lesion load along the tracts and syntactic processing, by calculating tract-lesion overlap for each patient and analyzing the correlation with syntactic data. Our findings revealed a Broca-striatum tract linking BA45 with the left caudate head and overlapping with VLSM voxel clusters relating to complex syntax. The lesion load values for this tract were correlated with complex syntax scores, whereas no such correlation was observed for the other tracts. These results extend current syntax-network models, by adding a deep "Broca-caudate pathway," and are consistent with functional accounts of frontostriatal circuits.

Aphasia or Neglect after Thalamic Stroke: The Various Ways They may be Related to Cortical Hypoperfusion

Although aphasia and hemispatial neglect are classically labeled as cortical deficits, language deficits or hemispatial neglect following lesions to subcortical regions have been reported in many studies. However, whether or not aphasia and hemispatial neglect can be caused by subcortical lesions alone has been a matter of controversy. It has been previously shown that most cases of aphasia or hemispatial neglect due to acute non-thalamic subcortical infarcts can be accounted for by concurrent cortical hypoperfusion due to arterial stenosis or occlusion, reversible by restoring blood flow to the cortex. In this study, we evaluated whether aphasia or neglect occur after acute thalamic infarct without cortical hypoperfusion due to arterial stenosis or occlusion. Twenty patients with isolated acute thalamic infarcts (10 right and 10 left) underwent MRI scanning and detailed cognitive testing. Results revealed that 5/10 patients with left thalamic infarcts had aphasia and only 1 had cortical hypoperfusion, whereas 2/10 patients with right thalamic infarcts had hemispatial neglect and both had cortical hypoperfusion. These findings indicate that aphasia was observed in some cases of isolated left thalamic infarcts without cortical hypoerfusion due to arterial stenosis or occlusion (measured with time-to-peak delays), but neglect occurred after isolated right thalamic infarcts only when there was cortical hypoperfusion due to arterial stenosis or occlusion. Therefore, neglect after acute right thalamic infarct should trigger evaluation for cortical hypoperfusion that might improve with restoration of blood flow. Further investigation in a larger group of patients and with other imaging modalities is warranted to confirm these findings.

Age-related changes in feature-based object memory retrieval as measured by event-related potentials

To investigate neural mechanisms that support semantic functions in aging, we recorded scalp EEG during an object retrieval task in 22 younger and 22 older adults. The task required determining if a particular object could be retrieved when two visual words representing object features were presented. Both age groups had comparable accuracy although response times were longer in older adults. In both groups a left fronto-temporal negative potential occurred at around 750ms during object retrieval, consistent with previous findings (Brier, Maguire, Tillman, Hart, & Kraut, 2008). In only older adults, a later positive frontal potential was found peaking between 800 and 1000ms during no retrieval. These findings suggest younger and older adults employ comparable neural mechanisms when features clearly facilitate retrieval of an object memory, but when features yield no retrieval, older adults use additional neural resources to engage in a more effortful and exhaustive search prior to making a decision.

Associative learning beyond the medial temporal lobe: many actors on the memory stage

Decades of research have established a model that includes the medial temporal lobe, and particularly the hippocampus, as a critical node for episodic memory. Neuroimaging and clinical studies have shown the involvement of additional cortical and subcortical regions. Among these areas, the thalamus, the retrosplenial cortex, and the prefrontal cortices have been consistently related to episodic memory performance. This article provides evidences that these areas are in different forms and degrees critical for human memory function rather than playing only an ancillary role. First we briefly summarize the functional architecture of the medial temporal lobe with respect to recognition memory and recall. We then focus on the clinical and neuroimaging evidence available on thalamo-prefrontal and thalamo-retrosplenial networks. The role of these networks in episodic memory has been considered secondary, partly because disruption of these areas does not always lead to severe impairments; to account for this evidence, we discuss methodological issues related to the investigation of these regions. We propose that these networks contribute differently to recognition memory and recall, and also that the memory stage of their contribution shows specificity to encoding or retrieval in recall tasks. We note that the same mechanisms may be in force when humans perform non-episodic tasks, e.g., semantic retrieval and mental time travel. Functional disturbance of these networks is related to cognitive impairments not only in neurological disorders, but also in psychiatric medical conditions, such as schizophrenia. Finally we discuss possible mechanisms for the contribution of these areas to memory, including regulation of oscillatory rhythms and long-term potentiation. We conclude that integrity of the thalamo-frontal and the thalamo-retrosplenial networks is necessary for the manifold features of episodic memory.

Functional roles of the thalamus for language capacities

Early biological concepts of language were predominantly corticocentric, but over the last decades biolinguistic research, equipped with new technical possibilities, has drastically changed this view. To date, connectionist models, conceiving linguistic skills as corticobasal network activities, dominate our understanding of the neural basis of language. However, beyond the notion of an involvement of the thalamus and, in most cases also, the basal ganglia (BG) in linguistic operations, specific functions of the respective depth structures mostly remain rather controversial. In this review, some of these issues shall be discussed, particularly the functional configuration of basal network components and the language specificity of subcortical supporting activity. Arguments will be provided for a primarily cortico-thalamic language network. In this view, the thalamus does not engage in proper linguistic operations, but rather acts as a central monitor for language-specific cortical activities, supported by the BG in both perceptual and productive language execution.

Thalamic mechanisms in language: a reconsideration based on recent findings and concepts

Recent literature on thalamic aphasia and thalamic activity during neuroimaging is selectively reviewed followed by a consideration of recent anatomic and physiological findings regarding thalamic structure and functions. It is concluded that four related corticothalamic and/or thalamocortical mechanisms impact language processing: (1) selective engagement of task-relevant cortical areas in a heightened state of responsiveness in part through the nucleus reticularis (NR), (2) passing information from one cortical area to another through corticothalamo-cortical mechanisms, (3) sharpening the focus on task-relevant information through corticothalamo-cortical feedback mechanisms, and (4) selection of one language unit over another in the expression of a concept, accomplished in concert with basal ganglia loops. The relationship and interaction of these mechanisms is discussed and integrated with thalamic aphasia and neuroimaging data into a theory of thalamic functions in language.

Functional imaging of the thalamus in language

Herein, the literature regarding functional imaging of the thalamus during language tasks is reviewed. Fifty studies met criteria for analysis. Two of the most common task paradigms associated with thalamic activation were generative tasks (e.g. word or sentence generation) and naming, though activation was also seen in tasks that involve lexical decision, reading and working memory. Typically, thalamic activation was seen bilaterally, left greater than right, along with activation in frontal and temporal cortical regions. Thalamic activation was seen with perceptually challenging tasks, though few studies rigorously correlated thalamic activation with measures of attention or task difficulty. The peaks of activation loci were seen in virtually all thalamic regions, with a bias towards left-sided and midline activation. These analyses suggest that the thalamus may be involved in processes that involve manipulations of lexical information, but point to the need for more systematic study of the thalamus using language tasks.

Visual Puzzles, Figure Weights, and Cancellation: Some Preliminary Hypotheses on the Functional and Neural Substrates of These Three New WAIS-IV Subtests

In this study, five consecutive patients with focal strokes and/or cortical excisions were examined with the Wechsler Adult Intelligence Scale and Wechsler Memory Scale-Fourth Editions along with a comprehensive battery of other neuropsychological tasks. All five of the lesions were large and typically involved frontal, temporal, and/or parietal lobes and were lateralized to one hemisphere. The clinical case method was used to determine the cognitive neuropsychological correlates of mental rotation (Visual Puzzles), Piagetian balance beam (Figure Weights), and visual search (Cancellation) tasks. The pattern of results on Visual Puzzles and Figure Weights suggested that both subtests involve predominately right frontoparietal networks involved in visual working memory. It appeared that Visual Puzzles could also critically rely on the integrity of the left temporoparietal junction. The left temporoparietal junction could be involved in temporal ordering and integration of local elements into a nonverbal gestalt. In contrast, the Figure Weights task appears to critically involve the right temporoparietal junction involved in numerical magnitude estimation. Cancellation was sensitive to left frontotemporal lesions and not right posterior parietal lesions typical of other visual search tasks. In addition, the Cancellation subtest was sensitive to verbal search strategies and perhaps object-based attention demands, thereby constituting a unique task in comparison with previous visual search tasks.

Subjective report of word-finding and memory deficits in normal aging and dementia

OBJECTIVE

Compare subjective reports of both memory and word-finding deficits to clinical diagnosis and objective neuropsychological testing.

BACKGROUND

With the increasing number of aging individuals with cognitive impairments, effective screening measures would improve the likelihood of detection. Subjective reports of symptoms are typically obtained in clinical settings, yet the validity of these reports is relatively unknown.

METHODS

Clinical screening for dementia was carried out at an Alzheimer disease center. Dichotomous ratings for memory and word-finding/language problems were given by patients and neurologists. These ratings were compared with 13 neuropsychological measures of word-finding/language and episodic memory.

RESULTS

Ratings of memory by both patients and neurologists correlated well with standard neuropsychological measures of memory. However, both the patients' and physicians' ratings of word-finding/language impairments had notably less of a correlation with the relevant neuropsychological measures of word-finding/language.

CONCLUSION

Compared with ratings of memory, similar assessments of word-finding/language difficulties were relatively inaccurate, and thus poor predictors of impairment. It is imperative to develop effective screening methods that will help reveal cognitive impairments, as this issue will almost certainly become more pressing given the projected increase in the number of aging individuals and those with dementia.

Foreseeing the future: occurrence probability of imagined future events modulates hippocampal activation

Episodic memory and episodic future thinking are known to share a set of brain regions. Potential differences in activation patterns associated with the two conditions are as yet inconclusive, in particular with respect to hippocampal involvement. Hippocampal activation is modulated by a range of phenomenal qualities during the imagination of both past and future events (Addis et al. (2004) Hippocampus 14:752-762; Addis and Schacter (2008) Hippocampus 18:227-237). A relevant variable in this regard is the occurrence probability of an episode, which varies for future but not past events and thus cannot be equated across conditions. Using parametric modulation analysis, we investigated the effect of occurrence probability of imagined future events on brain activation patterns, while effects of temporal distance, amount of details, and emotionality were controlled for. Activation of right anterior hippocampus increased with decreasing occurrence probability, presumably reflecting higher processing demands during binding of more disparate details for unlikely events. This finding may contribute to the understanding of previously reported inconsistent results concerning hippocampal involvement during the imagination of past and future events.

Broad and narrow conceptual tuning in the human frontal lobes

Previous work has implicated prefrontal cortices in selecting among and retrieving conceptual information stored elsewhere. However, recent neurophysiological work in monkeys suggests that prefrontal cortex may play a more direct role in representing conceptual information in a flexible context-specific manner. Here, we investigate the nature of visual object representations from perceptual to conceptual levels in an unbiased data-driven manner using a functional magnetic resonance imaging adaptation paradigm with pictures of animals. Throughout much of occipital cortex, activity was highly sensitive to changes in 2D stimulus form, consistent with tuning to form and position within retinotopic coordinates and matching an automated measure of shape similarity. Broad superordinate conceptual information was represented as early as extrastriate and posterior ventral temporal cortex. These regions were not completely invariant to form, suggesting that form similarity remains an important organizational constraint into the temporal cortex. Separate sites within prefrontal cortex represented broad and narrow conceptual tuning, with more anterior sites tuned narrowly to close conceptual associates in a manner that was invariant to stimulus form/position and that matched independent similarity ratings of the stimuli. The combination of broad and narrow conceptual tuning within prefrontal cortex may support flexible selection, retrieval, and classification of objects at different levels of categorical abstraction.

Lateral inferotemporal cortex maintains conceptual-semantic representations in verbal working memory

Verbal working memory, that is, the temporary maintenance of linguistic information in an activated state, is typically assumed to rely on phonological representations. Recent evidence from behavioral, neuropsychological, and electrophysiological studies, however, suggests that conceptual-semantic representations may also be maintained in an activated state. We developed a new semantic working memory task that involves the maintenance of a novel conceptual combination. Functional magnetic resonance imaging data acquired during the maintenance of conceptual combinations, relative to an item recognition task without the possibility of conceptual combination, demonstrate increased activation in the posterior left middle and inferior temporal gyri (known to be involved in conceptual representations) and left inferior frontal gyrus (known to be involved in semantic control processes). We suggest that this temporo-frontal system supports maintenance of conceptual information in working memory, with the frontal regions controlling the sustained activation of heteromodal conceptual representations in the inferior temporal cortex.

The timecourse of activation within the cortical network associated with visual imagery

The current study examined the hemodynamic timecourse of activation within a network of regions that is thought to be associated with visual imagery. Two experimental conditions were examined that were designed to place differential demands on specific nodes within the visual imagery network. The two tasks were an object inspection task and a mental rotation task. The two conditions recruited overlapping cortical regions; however several regions revealed a differential response to object inspection and mental rotation. The mental rotation condition elicited greater activation in parietal cortex, lateral occipital/temporal regions, and bilateral prefrontal cortex. Conversely, the object inspection condition elicited greater activation in inferior extrastriate cortex, the inferior frontal gyrus, and the right cerebellum. When examining the timecourse of activation three different timecourse patterns were observed across cortical regions and conditions. The shape of the hemodynamic timecourse appears to correspond strongly with the cognitive processing taking place within the region, not the stimulus paradigm. The paper discusses the significance of those varying timecourse shapes and has implications for the appropriateness of using the canonical hrf during fMRI data analysis.

Common and Unique Neural Activations in Autobiographical, Episodic, and Semantic Retrieval

This study sought to explore the neural correlates that underlie autobiographical, episodic, and semantic memory. Autobiographical memory was defined as the conscious recollection of personally relevant events, episodic memory as the recall of stimuli presented in the laboratory, and semantic memory as the retrieval of factual information and general knowledge about the world. Our objective was to delineate common neural activations, reflecting a functional overlap, and unique neural activations, reflecting functional dissociation of these memory processes. We conducted an event-related functional magnetic resonance imaging study in which we utilized the same pictorial stimuli but manipulated retrieval demands to extract autobiographical, episodic, or semantic memories. The results show a functional overlap of the three types of memory retrieval in the inferior frontal gyrus, the middle frontal gyrus, the caudate nucleus, the thalamus, and the lingual gyrus. All memory conditions yielded activation of the left medial-temporal lobe; however, we found a functional dissociation within this region. The anterior and superior areas were active in episodic and semantic retrieval, whereas more posterior and inferior areas were active in autobiographical retrieval. Unique activations for each memory type were also delineated, including medial frontal increases for autobiographical, right middle frontal increases for episodic, and right inferior temporal increases for semantic retrieval. These findings suggest a common neural network underlying all declarative memory retrieval, as well as unique neural contributions reflecting the specific properties of retrieved memories.

Spontaneous activity associated with primary visual cortex: a resting-state FMRI study

Brain functions during the resting state have attracted considerable attention in the past several years. However, little has been known about spontaneous activity in the sensory cortices in the task-free state. This study used functional magnetic resonance imaging (fMRI) to investigate the existence of spontaneous activity in the primary visual areas (PVA) of normal-sighted subjects and to explore the physiological implications of such activity. Our results revealed that we were able to detect spontaneous activity, which was nonrandom in that it was distinctly clustered both temporally and spatially in the PVA of each subject. In addition, the neural network associated with the PVA-related spontaneous activity included the visual association areas, the precuneus, the precentral/postcentral gyrus, the middle frontal gyrus, the fusiform gyrus, the inferior/middle temporal gyrus, and the parahippocampal gyrus. After considering the functions of these regions, we speculated that the PVA-related spontaneous activity may be associated with memory-related mental imagery and/or visual memory consolidation processes. These findings confirm the presence of spontaneous activity in the PVA and related brain areas. This confirmation supports the perspective that brain is a system intrinsically operating on its own, and sensory information interacts with rather than determines the operation of the system.

The Semantic Object Retrieval Test (SORT) in Amnestic Mild Cognitive Impairment

BACKGROUND

Between 10% and 15% of patients with the amnestic variety of Mild Cognitive Impairment (MCI) convert to Alzheimer disease (AD) per year.

OBJECTIVE

Characterize cognitive markers that may herald conversion from MCI to AD and directly assess semantic memory in patients meeting criteria for amnestic MCI.

DESIGN

Thirty-five amnestic MCI patients and 121 healthy aging controls enrolled at an Alzheimer Disease Center received a battery of standard neuropsychologic tests, and the Semantic Object Retrieval Test (SORT), a test that we have developed for the assessment of semantic memory and subsequent name production, and that has been shown to be able to differentiate between normals and patients with AD.

RESULTS

On the basis of normative data from the SORT, the MCI subjects could be divided into 2 groups: 10 patients (29%) with a significant semantic impairment (SI+) and 25 without a semantic memory deficit (SI-). There was a significant correlation between all SORT variables and performance on the Boston Naming Test. In this MCI population, significantly impaired SORT performance was associated with a relative decrease in performance on tests of frontal lobe functions, although disruption of thalamic-related processes cannot be excluded as an etiology for semantic memory impairment.

CONCLUSIONS

The SORT is a specific test of semantic memory, and is a sensitive measure of semantic memory deficits in patients who otherwise meet criteria for amnestic MCI. Using this specific assessment tool, a significant number of MCI patients were found to have semantic memory deficits. As these patients may be early in the course of possible progression toward dementia, the SORT or other tests of semantic memory may provide important diagnostic or prognostic information in patients with MCI.

Deriving meaning: Distinct neural mechanisms for metaphoric, literal, and non-meaningful sentences

In this study, we used a novel cognitive paradigm and event-related functional magnetic resonance imaging (ER-fMRI) to investigate the neural substrates involved in processing three different types of sentences. Participants read either metaphoric (Some surgeons are butchers), literal (Some surgeons are fathers), or non-meaningful sentences (Some surgeons are shelves) and had to decide whether they made sense or not. We demonstrate that processing of the different sentence types relied on distinct neural mechanisms. Activation of the left inferior frontal gyrus (LIFG), BA 47, was shared by both non-meaningful and metaphoric sentences but not by literal sentences. Furthermore, activation of the left thalamus appeared to be specifically involved in deriving meaning from metaphoric sentences despite lack of reaction times differences between literals and metaphors. We assign this to the ad hoc concept construction and open-endedness of metaphoric interpretation. In contrast to previous studies, our results do not support the view the right hemispheric is specifically involved in metaphor comprehension.

Neural correlates of the object-recall process in semantic memory

The recall of an object from features is a specific operation in semantic memory in which the thalamus and pre-supplementary motor area (pre-SMA) are integrally involved. Other higher-order semantic cortices are also likely to be involved. We used the object-recall-from-features paradigm, with more sensitive scanning techniques and larger sample size, to replicate and extend our previous results. Eighteen right-handed healthy participants performed an object-recall task and an association semantic task, while undergoing functional magnetic resonance imaging. During object-recall, subjects determined whether words pairs describing object features combined to recall an object; during the association task they decided if two words were related. Of brain areas specifically involved in object recall, in addition to the thalamus and pre-SMA, other regions included the left dorsolateral prefrontal cortex, inferior parietal lobule, and middle temporal gyrus, and bilateral rostral anterior cingulate and inferior frontal gyri. These regions are involved in semantic processing, verbal working memory and response-conflict detection and monitoring. The thalamus likely helps to coordinate activity of these different brain areas. Understanding the circuit that normally mediates this process is relevant for schizophrenia, where many regions in this circuit are functionally abnormal and semantic memory is impaired.

Material-dependent activation in prefrontal cortex: working memory for letters and texture patterns--initial observations

PURPOSE

To prospectively evaluate whether a distinction between verbal and nonverbal short-term memory systems, as predicted with the multicomponent working memory model, is reflected in the material-specific patterns of activation in the prefrontal cortex.

MATERIALS AND METHODS

Informed written consent was obtained from all participants, and the institutional review board approved the study protocol. Echo-planar MR imaging was performed in 12 healthy subjects (five female and seven male subjects), with a mean age of 23.52 years +/- 2.52 (standard deviation) and a range of 20-29 years. A two-back task was used in the verbal and nonverbal versions. In the first version, letters were used as stimuli, and in the second version, the stimuli were abstract texture patterns. Timing parameters for both versions were the same. Statistical analysis of the functional data involved a fixed-effects general linear model. Regions of activation were identified from specific t-statistic contrasts between baseline and active tasks (corrected for whole-brain multiple comparisons).

RESULTS

The following suprathreshold voxels for the verbal condition were observed predominantly in the left hemisphere (middle frontal gyrus, precentral gyrus, middle temporal gyrus, and occipital cortex). Bilateral activations were in inferior frontal gyri, insulae inferior, superior parietal lobules, and cingulate gyri. In the nonverbal condition, suprathreshold voxels were located mostly bilaterally in the following regions: inferior, middle, and medial frontal gyri and inferior parietal lobules. Active regions were also found in the precentral gyrus and precuneate gyrus in the left parietal lobe and the occipital cortex in the right hemisphere.

CONCLUSION

Results of this study are consistent with the multicomponent model of working memory.

Neural mechanisms of semantic memory

Using previously published functional magnetic resonance imaging work studying object recall as a framework, we describe the spatial and temporal properties of brain activation as one plausible model of visually triggered access to semantic memory. We suggest that interactions between the dorsomedial thalamus and Brodmann area 6 facilitate the setting of object search criteria and perhaps drive the search. The pulvinar nucleus of the thalamus, in concert with ventral temporal and occipital cortical regions, become active later, perhaps when the criteria are met or the target object is identified.

fMRI differences in encoding and retrieval of pictures due to encoding strategy in the elderly

Functional MRI (fMRI) was used to examine the neural correlates of depth of processing during encoding and retrieval of photographs in older normal volunteers (n = 12). Separate scans were run during deep (natural vs. man-made decision) and shallow (color vs. black-and-white decision) encoding and during old/new recognition of pictures initially presented in one of the two encoding conditions. A baseline condition consisting of a scrambled, color photograph was used as a contrast in each scan. Recognition accuracy was greater for the pictures on which semantic decisions were made at encoding, consistent with the expected levels of processing effect. A mixed-effects model was used to compare fMRI differences between conditions (deep-baseline vs. shallow-baseline) in both encoding and retrieval. For encoding, this contrast revealed greater activation associated with deep encoding in several areas, including the left parahippocampal gyrus (PHG), left middle temporal gyrus, and left anterior thalamus. Increased left hippocampal, right dorsolateral, and inferior frontal activations were found for recognition of items that had been presented in the deep relative to the shallow encoding condition. We speculate that the modulation of activity in these regions by the depth of processing manipulation shows that these regions support effective encoding and successful retrieval. A direct comparison between encoding and retrieval revealed greater activation during retrieval in the medial temporal (right hippocampus and bilateral PHG), anterior cingulate, and bilateral prefrontal (inferior and dorsolateral). Most notably, greater right posterior PHG was found during encoding compared to recognition. Focusing on the medial temporal lobe (MTL) region, our results suggest a greater involvement of both anterior MTL and prefrontal regions in retrieval compared to encoding.

Neural hybrid model of semantic object memory: implications from event-related timing using fMRI

Previous studies by our group have demonstrated fMRI signal changes and synchronized gamma rhythm EEG oscillations between thalamus and cortical regions as subjects recall objects from visually presented features. Here, we extend this work by estimating the time course of fMRI signal changes in the cortical and subcortical regions found to exhibit evidence for task-related activation. Our results indicate that there are separate loci of signal changes in the thalamus (dorsomedial and pulvinar) that exhibit notable differences in times of onset, peak and return to baseline of signal changes. The signal changes in the pulvinar demonstrate the slowest transients of all the cortical and subcortical regions we examined. Evaluation of cortical regions demonstrated salient differences as well, with the signal changes in Brodmann area 6 (BA6) rising, peaking, and returning to baseline earlier than those detected in other regions. We conclude that BA6 mediates early designation or refinement of search criteria, and that the pulvinar may be involved in the binding of feature stimuli for an integrated object memory.

The neural substrate of analogical reasoning: an fMRI study

This study investigated the anatomical substrate of analogical reasoning using functional magnetic resonance imaging. In the study, subjects performed a verbal analogy task (e.g., soldier is to army as drummer is to band) and, to control for activation caused by purely semantic access, a semantic judgment task. Significant activation differences between the verbal analogy and the semantic judgment task were found bilaterally in the prefrontal cortex (right BA 11/BA 47 and left BA45), the fusiform gyrus, and the basal ganglia; left lateralized in the postero-superior temporal gyrus (BA 22) and the (para) hippocampal region; and right lateralized in the anterior cingulate. The role of these areas in analogical reasoning is discussed.

Interactions between thalamic and cortical rhythms during semantic memory recall in human

Human scalp electroencephalographic rhythms, indicative of cortical population synchrony, have long been posited to reflect cognitive processing. Although numerous studies employing simultaneous thalamic and cortical electrode recording in nonhuman animals have explored the role of the thalamus in the modulation of cortical rhythms, direct evidence for thalamocortical modulation in human has not, to our knowledge, been obtained. We simultaneously recorded from thalamic and scalp electrodes in one human during performance of a cognitive task and found a spatially widespread, phase-locked, low-frequency rhythm (7-8 Hz) power decrease at thalamus and scalp during semantic memory recall. This low-frequency rhythm power decrease was followed by a spatially specific, phase-locked, fast-rhythm (21-34 Hz) power increase at thalamus and occipital scalp. Such a pattern of thalamocortical activity reflects a plausible neural mechanism underlying semantic memory recall that may underlie other cognitive processes as well.